International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 16, No. 05, 2022 Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications for Children Road Safety Education https://doi.org/10.3991/ijim.v16i05.26359 Nur Sauri Yahaya(*), Ariffin Abdul Mutalib, Sobihatun Nur Abdul Salam School of Multimedia Technology and Communication, Universiti Utara Malaysia, Kedah, Malaysia nursauri@uum.edu.my Abstract—Nowadays, developing Virtual Reality (VR) and Immersive Virtual Reality (IVR) application is one of the alternative support tools used to support children in learning road safety education. At the early stage of VR/IVR application development, cybersickness was one of the main concerns that occur in both non-immersive and immersive environments especially with a Head- Mounted Display (HMD) equipped system that is mainly utilized in mobile and tethered VR/IVR platforms. While HMDs became available as consumer products, manufacturers struggled with hardware limitations that might induce cybersickness, due to inconsistencies between visual and physical movement in the virtual environment. Previous research has examined the factors, symptoms, and guidelines in reducing cybersickness and has been shown to minimize the effect of cybersickness. However, the studies on cybersickness reduction guide- lines in VR/IVR application for Children Road Safety Education (CRSE) have not been studied much. Thus, there are two objectives of this study; i) to explore and identify appropriate cybersickness reduction guidelines in developing VR/ IVR applications, and ii) to propose the design elements and principles of cyber- sickness reduction guidelines. The findings of this study outline the appropriate design elements and principles of cybersickness reduction guidelines that are applicable and beneficial in the development process of the VR/IVR application and respectively useful in reducing the effect of cybersickness. Keywords—virtual reality, immersive virtual reality, cybersickness, simulator sickness, children road safety 1 Introduction VR is an interactive, immersive, and realistic three-dimensional computer-simulated world that provides a significant learning experience [1]. According to [2], the develop- ment of new interactive technologies inevitably has an impact on all aspects of teaching and learning by increasing student motivation and participation in the learning process. It was claimed as a result of the improvement in VR interfaces, interaction techniques, iJIM ‒ Vol. 16, No. 05, 2022 33 https://doi.org/10.3991/ijim.v16i05.26359 mailto:nursauri@uum.edu.my Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… and devices in providing more natural and apparent modes of interaction and motiva- tional elements [3]. On the other hand, an interactive learning environment also has successfully enriched the ease of use, flexibility, and effectiveness in teaching and learning experience [4]. In the context of children’s road safety education, the imple- mentation of VR has begun in the past sixteen years ago [5]–[9]. CRSE is a lifelong learning process that improves the quality of knowledge, and it should begin at a young age [10]. Extending its message to the audience to teach good road safety behaviors and habits needs to start from primary to secondary school levels so that the knowledge can become ingrained as part of the culture and practiced among children [11]. Besides its powerful 3-dimensional representation, VR is also known as a potential instructional tool that provides simulated training and skills especially in dangerous and difficult situations, which can become harmful to children when they are learning road safety education [1]. By allowing users to interact with alternate realities generated by com- puters such as the Virtual Environment (VE), it will help them to perform according to the assigned activities and influence them in producing appropriate outcomes from the virtual activities in the VE. In VE, users do not operate computer applications via an interface. Instead, peo- ple participate, they perform tasks, and they experience activities within a computer- generated world [12]. To achieve a higher immersion level when interacting with the VE, the HMD allows users to have a better experience in IVR. HMD can be divided into two categories which are Mobile and Tethered HMD. Mobile HMD such as Sam- sung Gear, Google Cardboard, and Google DayDream is a holder with lenses, which users use to place smartphones. The lenses divide the screen into two images and turn the smartphone into a VR device. It is relatively inexpensive and does not require any wire to connect to the devices but is still capable of providing accurate three-degree-of- freedom (3DoF) but not six-degree-of-freedom (6DoF) motion tracking. On the other hand, tethered HMD is a device that is capable to track the motion of users accurately through 6DoF. It is physically connected to PCs and other output devices like LCD monitors and audio speakers. Furthermore, the use of dedicated displays and built-in motion sensors inside Teth- ered HMD devices drastically improves the image fidelity and motion tracking of the VR system. The HMD Oculus Rift and HTC Vive are two examples of tethered HMD devices that provide stereoscopic 3D display, which is worn on the head or a part of the helmet that provides an immersive virtual reality experience [13]. It is a new form of interaction between humans and machines with full visual immersion that is beyond the use of a keyboard, mouse, and touch-screen. However, at the early stage of VR simula- tion development, the utilization of HMD with the VR application has simulator sick- ness and cybersickness, which rise huge concern in VR application development. This problem occurs in both non-immersive and immersive VR and also for both mobile and tethered platforms as they both integrate visual and physical movement in the VE. Thus, to minimize the effect of cybersickness, several guidelines and design principles have been identified, which are suitable to be adopted in designing the VR/IVR appli- cations for both platforms. From that standpoint, there are two objectives of this study which is: i) exploring and identifying appropriate cybersickness reduction guideline in VR/IVR applications 34 http://www.i-jim.org Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… employed in previous research, and ii) proposing the design elements and principles of cybersickness reduction guidelines. The first section provides a brief introduction of the selected topic, which briefly describes the VR, IVR, CRSE, VE, and HMD. The next section extends on the concept, causing factors and some theories are related to cybersickness. Then, the methodology section explains the process of the comparative analysis conducted in identifying appropriate cybersickness reduction guidelines from 14 selected studies. This review contributes to the identification and justification of the selected cybersickness reduction guidelines (Table 3). The next section elaborates the detailed prosed design elements of cybersickness reduction guidelines which are then validated through the expert review process. Then, it is followed by the findings of the study with some discussions related to the comments and suggestions from the expert which are useful in improvising the proposed design elements and principles of cyber- sickness reduction guidelines. Finally, the conclusion section summarizes the whole content of the paper. 2 Cybersickness Cybersickness happens in a situation where users use any particular part of their body as a stationary and perform a specific movement, but the physical movement performed is not aligned with the visual representation in the virtual environment [14]. Meanwhile, simulator sickness refers to the symptoms of discomfort affected through the use of VE such as discomfort, disorientation, fatigue, sweating, nausea, vomiting, and headaches [12], [15]. These symptoms reduce the effectiveness of the simulators for virtual training and result in decreased simulator interest, or adaptation of inappro- priate mechanisms during the training session. On top of that, post-effect on having the simulator sickness and cybersickness can directly impact the individuals, like drowsi- ness or postural instability sickness. In regards to that, [16] has identified several fac- tors possibly causing both sicknesses, such as acceleration, degree of control, duration of use, altitude, field of view (FOV), latency, distortion correction, flicker and experi- ence. Besides, with the advancement of a VR/IVR technology, many researchers and companies have provided some initiatives or guidelines in designing VR/IVR applica- tion to reduce the cybersickness based on the factor which has been identified in the previous studies [13], [14], [16]–[18]. As mentioned by [16] the guidelines basically focus on the software development aspect by using HMD as an interaction tool in the virtual environment. Other than symptoms and factors, there are some prominent theories causing cybersickness that should be considered by researchers before developing VR or IVR applications in both mobile and tethered platforms. According to [12] and [19], both mentioned in their study that there are three main categories known as the causes of cybersickness, which are the poison theory, postural instability theory, and sensory con- flict theory. The first theory involves a conflict between the vestibular system, visual and other sensory input systems where the response does not consistent with the sub- ject [20]. The second theory is developed by [21], where the uncontrolled movement of the perception and system actions is based on the process of maintaining postural iJIM ‒ Vol. 16, No. 05, 2022 35 Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… stability in the environment. In other words, the longer the postural instability in the environment, it will result in more severe symptoms of cybersickness [15]. The last cybersickness theory is referred to as the influence between two main sensory systems involved in the VE, namely visual and vestibular senses. Both senses provide mis- matched information about individual orientation and perceived motion which leads to misunderstanding for the human body that thinks it has ingested some type of toxic substance thus causing symptoms of emetic response [19]. Thus, to develop a VR or IVR application that can minimize the effect of cybersick- ness for both platforms, a proper cybersickness reduction guideline and design princi- ple should be identified and extracted before designing the application. Gradually, there are researchers, and IVR manufacturers begin incorporating cybersickness reduction guideline that improves the dynamic focus solution to reduce discomfort in the immer- sive VE [13], [14], [16]–[18]. Accordingly, several guidelines have been identified and extracted from the comparative analysis, and the detail of the element are described in the next section. 3 Study method A comparative analysis was conducted to identify the appropriate cybersickness reduction guidelines that applicable for VR/IVR application development. Once the cybersickness reduction guidelines are identified, a set of design elements and princi- ples of cybersickness reduction guidelines are proposed. An expert review session was conducted to validate the proposed design elements and principles of cybersickness reduction guidelines. 3.1 Comparative analysis This study has been conducted qualitatively through a comparative analysis tech- nique. The process begins by performing a literature search by using Mendeley refer- encing tool and Google Scholar, Research Gate, and ACM Digital Library as web-based search platforms that are beneficial in returning relevant search results via numerous global journals, proceedings, and online databases. The scope of the literature search is mainly focused on VR, IVR, CRSE, and cybersickness design guidelines. To retrieve relevant results, the following most relevant keywords have been used for searching purposes: “cybersickness guideline”, “simulator sickness guidelines”, “cybersickness and simulator sickness symptom and effect”, “virtual reality application”, “immersive virtual reality” and “children road safety education”. At the beginning of the literature search, 100 studies have identified through online database searching based on the pre-defined themes. From the retrieved result, there are 14 studies considered [12], [14], [16], [22]–[30]. All of the selected studies were manually elicited and filtered by using a specific pre-defined theme: “cybersickness reduction design guidelines” “virtual reality”,” immersive virtual reality” and “chil- dren road safety education”, to obtain appropriate and relevant studies for this study. 36 http://www.i-jim.org Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… As mentioned earlier, this process was mainly conducted by using Mendeley, as it quickly supports in terms of the searching operator and managing the retrieved article to the targeted folder. All selected articles were reviewed and analyzed based on their relevance in-term of cybersickness reduction guidelines and design principles in devel- oping the application. Although the scope and the context in those studies are different, the similarities between the guidelines in reducing cybersickness effects are suitable for this study. 3.2 The proposed design element of cybersickness reduction guidelines Cybersickness reduction element is very important and is highly recommended to be included for any IVR application development. From the previous study analysis, most of the VR and IVR applications neglect the importance of cybersickness reduction. As a result, it leads to discomfort symptoms such as nausea, sickness, and headaches to some people. Besides, lack of cybersickness guidelines is also one of the factors that prevent researchers from developing the VR or IVR application that has had minimal effect on cybersickness. Gradually, there are researchers, and IVR manufacturers begin incorporating cybersickness reduction guideline that improves the dynamic focus solu- tion to reduce discomfort in the immersive virtual environment [13], [14], [16]–[18]. The result from the analysis, as mentioned above, are exhibited in Table 1. It shows the summary of the appropriated and selected cybersickness reduction guidelines, which are beneficial to be adapted in the development process of the VR/IVR appli- cations. Meanwhile, Table 2 lists the detailed description of the basic indicators that are used to determine the condition of the selected cybersickness reduction guideline. Two themes have been used to classify the previous studies, i.e. application and model, framework, and guidelines. The main reason for categorizing these studies is to sim- plify the classification process of the analyzed data and improve folder organization. Referring to the analysis, 12 appropriate guidelines have been identified and extracted, which are the duration of use, experience, physical setting, acceleration speed, movement speed, a field of view, degree of control, camera, spatial sound, head tracking view, positional tracking, and persistence. All the listed guidelines are appropriate in the development of VR and IVR applications for both platforms whether mobile and tethered that cover the common factors that contribute to the cybersickness aspect. Besides, having extracted from the reviewed process, eight guidelines show a higher percentage (50% and greater) of the importance of the reduction guideline in addressing cybersickness. In contrast, the rest have a score of below 50%. Thus, eight of the guidelines are recommended to be adapted in VR and IVR applications. The other four still can be considered based on the requirement of the research project. Overall, all of the listed guidelines have their functionality and characteristic that will support the design process in minimizing discomforts in the simulator and cybersick- ness in both platforms. iJIM ‒ Vol. 16, No. 05, 2022 37 Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… Table 1. Extracted cybersickness reduction guideline from previous studies Categories Application Model/Framework/Guidelines Total (%)No. Study Guidelines 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Duration of Use Y Y Y Y Y Y Y 7 50 Experience Y Y Y Y Y Y Y Y 8 57 Physical Setting Y Y Y Y Y 5 37 Acceleration Speed Y Y Y Y Y Y Y Y Y Y Y Y 12 86 Movement Speed Y Y Y Y Y Y Y Y Y Y 10 71 Field of View Y Y Y Y Y Y Y Y Y Y 10 71 Degree of Control Y Y Y Y Y Y Y 7 50 Camera Y Y Y Y Y 5 37 Spatial Sound Y Y Y Y Y Y 6 43 Head Tracking View Y Y Y Y Y Y Y 7 50 Positional Tracking Y Y Y 3 22 Persistence Y Y Y Y Y Y Y Y 8 57 Note: Y refers to YES. Table 2. Indicator to determine the selected cybersickness reduction guidelines Indicator Description Condition of Classification Compulsory (C) Majority study There are between 8 to 14 models that apply the component (50%–100%) Recommended (R) Few study There are between 1 to 7 models that apply the component (1%–49%) From the comparative analysis result, each of the guidelines has its description and design elements that help to minimize the effect of cybersickness toward VR/IVR application development. The first guideline involves two elements which are (i) pro- viding a limited time of playing, and (ii) providing notification for resting time. In the second guideline, there are three elements involved which are, (i) the process of giv- ing proper exposure on IVR devices, (ii) providing understandable and straightforward instructions, and (iii) providing a continuation of assistance inside the VE [13], [14], [16]. In the physical setting, (i) provide a small screen/viewing area, (ii) provide com- fortable playing area, and (iii) proper sitting position suggestion is a design element that improves user experience and at the same time capable of reducing the effect of cyber- sickness [28], [31]. The fourth and fifth guidelines are about the adjustment of acceler- ation and movement speed inside the VE. The design elements for acceleration speed guidelines consist of (i) providing lower or slow acceleration movement, (ii) making acceleration short and infrequent, (iii) providing constant acceleration speed, (iv) allowing the user to control and initiate acceleration speed, while design element for movement speed guidelines are (i) provide constant velocity in character movement, (ii) provide realistic and straightforward body movement and, (iii) provide useful visual 38 http://www.i-jim.org Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… cues to users [13], [14], [16], [22], [29], [30]. In the FOV guideline, the design elements are about the process of changing the distance of viewing inside the VE [13], [14], [16]. The seventh guideline refers to the user degree of control, which consists of two appropriate design elements: synchronization of the camera movement and availability of avatar gesture inside the application [13], [14], [16]. Camera guidelines involved three main design elements such as (i) avoiding the zooming camera effect, (ii) avoid- ing unnecessary camera templates, and (iii) avoiding using head bobbing effect that can create discomforts to the user when interacting inside the VE. In terms of sound guidelines, providing positional tracking with audio design and offering spatializing audio is one of the design elements that are suitable to be adapted in minimizing the effect of cybersickness [16]. Head viewpoint and positional tracking guidelines are also important, especially in tracking user movement inside the virtual environment. The design elements for both types of tracking included (i) provide responsive user movement display, (ii) use SDK’s position tracking and head model, while design ele- ments for positional tracking guidelines involved (i) provide suitable starting position or location, (ii) limit user is movement/collider component, and (iii) provide positional tracking warning sign [13], [16]. The last design elements for cybersickness reduction guidelines refer to the persistence of the movement for each character, 3D model, and information transition inside the application [13], [25], [26]. Any movement and tran- sition of information inside VE must be persistent and consistent. The persistence of every character movement and viewing transition can help the user to maintain their focus in performing activities inside the virtual environment. Although the guidelines are appropriate for both platforms, concerning the mobile platform aspect, some of the identified guidelines are still needed consideration in terms of immersion and presence, navigation, and interaction aspect before they can be utilized in the development of mobile VR applications. For example, four identified guidelines such as duration of use, experience, physical setting, and sound effect are suitable to be utilized since they required minimum immersive interaction and naviga- tion inside the virtual environment meanwhile other guidelines are not suitable because they required physical interaction and also immersive interaction and navigation inside the virtual environment. 3.3 The validation process of the proposed design element of cybersickness reduction guidelines To validate the proposed design elements of cybersickness reduction guidelines, an expert review process was conducted. The process was conducted quantitively by involving four experts. They are academics and practitioners who have worked in the area related to virtual reality and its user experience for at least 5 years. As mentioned by [32]–[34]the total number of the agreed experts is sufficient enough to generate specific feedback and suggestion for the particular selected topic. 3.4 Participant profile and setting The experts are selected from different fields and expertise such as VR, Augmented Reality (AR), IVR, Visual Design, and Educational Technology. The experts must have iJIM ‒ Vol. 16, No. 05, 2022 39 Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… an academic qualification in Master or Ph.D. and have experience for at least five years. In accordance, Table 3 lists the demographic profile of the experts who have agreed to participate in his study. The listed expert, basically a person who hold a Ph.D. degree, represent the different field of expertise, knowledge, and experience from local and international institutions. The diversity of the selected experts is important for this study to obtain specific feedback and suggestion, which can improve the proposed design ele- ments for cybersickness sickness reduction guidelines. The rationale of selecting those experts is based on their achievement, involvement, and contribution in their respective fields, which is indirectly related to this study. Table 3. Demographic profile of the experts Expert Gender Education Level Field of Expertise Years of Experience A Female PhD VR >10 Years B Male PhD VR/UCD >5 Years C Male PhD VR/EdTech >7 Years D Male PhD VR >5 Years Notes: UCD = User-Centered Design; EdTech = Educational Technology. 3.5 Data collection procedure and instrument Due to Coronavirus (Covid-19) pandemic, most of the communication mediums used for this review process were through email, phone calls, and online meetings. The process started by identifying the expert through the official university expert website. The expert’s search criteria are based on their expertise, as mentioned in the previous section. Once the experts have been identified, an invitation email was sent to them. By agreeing to participate in the review process, the experts were appointed as Expert Reviewers in this study by sending them an appointment letter and a consent form. After a signed and stamped consent form has been received from the experts, a set of review instruments was emailed to the appointed expert for their further action. The expert was given around three to four weeks to complete the instrument form and required to return the form by using email. The review instrument used for this study is adapted from a previous study which is [34], [35] that consists of three main sections. The first section describes the detailed instruction and guidelines for the reviewer. The second section gathers the appropri- ate demographic data of the expert reviewers, which include their name, age, gender, level of education, area of expertise, and experience. The last section of the instrument consists of four main items of review questions, which are (i) the relevancy and under- standability of the design element and principle of the cybersickness reduction guide- lines ((ii) the understandability of the term used, (iii) the logical relationship and flow of all proposed design elements and principles, and (iv) the readability of the proposed design elements and principles. The optional items are provided to the reviewer, which they can provide suggestions and comments that can add extra insight and values to the proposed model. There are three options of answers for the reviewer to tick or select, which are “some are NOT relevant” and “need very detail explanation”, “some may 40 http://www.i-jim.org Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… be NOT relevant” and “need some explanation”, and “all are relevant” and “easy to understand”. 4 Result and discussion In this section, all the collected data from the expert review process are tabulated in Table 4. This included the frequency of response (n = 4) of the suitability and relevan- cies of the instrument in retrieving the data from the experts. Additionally, some experts provide additional comments and suggestions in reviewing the proposed elements which mean some of the proposed design elements need clarification and justification in term of suitability and applicability. As elaborated in the above table, the majority of the expert agrees that eight guidelines (21 elements) are relevant and the description of each design element is easy to understand while the other 4 guidelines (8 elements) are maybe not relevant and need some explanation and justification to be included as a proper design element for cybersickness reduction guidelines. Besides, the majority of the experts agree that the term, logical relationship between the design elements of cybersickness reduction guidelines is understandable and readable. However, there is only one expert found difficulties in understanding and reading the proposed design guideline of cybersickness reduction guidelines, which require extra attention on the readability aspect. Table 4. Frequencies of feedback from the expert review process Instrument Items Frequency (n=4) Q1 The proposed design elements and principles of cybersickness reduction guidelines are relevant and easily understood Some are NOT relevant Some maybe NOT relevant All are relevant Need very detailed explanation Need some explanation Easy to understand Duration of Use (2 Elements) Experiences (3 Elements) Physical Setting (3 Elements) Acceleration Speed (4 Elements) Movement Speed (3 Elements) FOV (1 Element) Degree of Control (2 Elements) Camera (3 Elements) Sound Effect (2 Elements) Head Tracking Viewpoint (2 Elements) Positional Tracking (3 Elements) Persistence (1 Element) 0 0 0 0 0 0 0 0 0 0 0 0 2 1 1 1 1 2 1 2 2 1 1 1 2 3 3 3 3 2 3 2 2 3 3 3 Yes No Q2 The term use is easy to understand 4 0 Q3 The relationship and flows of all main components and sub-component are logical 3 1 Q4 Overall, the design model is readable 3 1 Note: Q = Question. iJIM ‒ Vol. 16, No. 05, 2022 41 Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… For the additional comment and suggestion section, there is only one expert from a total of four who suggest making some adjustment to a particular design element of cybersickness reduction guidelines which is acceleration speed and movement speed. Expert D suggested that both sections need to be removed from the list of proposed guidelines and focus on those that directly influence the design e.g., degree of control, head tracking viewpoint, and positional tracking. In response to that suggestion, this study retained both design elements because these elements were retrieved and extracted from the original manufacturer sources such as HTC, Epic Games, Oculus as well from VR/IVR prominent scholar who mentioned acceleration and movement speed is one of important design elements in reducing cybersickness in VR/IVR application [14], [16]–[18], [27]. Based on the clarification and justification of the mentioned comment from the experts, the design element of cybersickness reduction guidelines was revised and updated accordingly. Table 5 shows the detailed description of the validated design elements of cybersickness reduction guidelines. Table 5. Validated design elements and principle for cybersickness reduction guidelines Guidelines Design Elements and Principle Description Duration of Use Provide limited time for the duration of use. It is suggested to take a break after every 15–30 minutes of playing time. But if the user feels the cybersickness as the early stage of playing time, it is suggested to stop playing temporarily before starting a new session of playing. Provide the notification for resting time. The application needs to have the capability of alerting or notifying the user to take a break periodically every time they are using the application so that discomfort symptoms can be avoided. Experience Provide proper exposure for the user before using IVR devices. Users should be exposed to the information on using IVR devices such as handling HMD controllers and familiarizing themselves with the play area. Provide simple and understandable instruction or guidelines for the beginner. Instruction or guidelines on using IVR devices should be prepared and explained to users before they use the application. The instruction or guideline must be clear and easy to understand Provide continuous assistance when they are using the IVR devices. Users need to be continuously guided and assisted when they are using the IVR devices so that they can perform the activities in the VE Physical Setting Provide a small screen/ viewing area. A small screen area is sufficient in displaying the VE. This is because the user will use HMD to view the VE, so it indirectly reduces the spaces of the playing area Provide a comfortable playing area. The playing area should be created comfortably without distracting user movement inside the playing area. This refers to the position of the motion tracking device, computer monitors, and other equipment A sitting position is suggested. A sitting position is suggested for users who do not have any experience using IVR. This is because they need to familiarize themselves with the VE with the actual environment (Continued) 42 http://www.i-jim.org Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… Guidelines Design Elements and Principle Description Acceleration Speed Provide lower or slow acceleration movement. Any acceleration movement speed in the VE is suggested to be in lower or slower condition. For instance, acceleration speed will make some people feel uncomfortable. So, lowering acceleration will generate a more comfortable experience for the user. Make acceleration short and infrequent. Unnecessary acceleration can cause a mismatch among visual, vestibular, and proprioceptive senses. So, by reducing the duration and frequency of acceleration it can minimize the effect of cybersickness. Provide constant acceleration speed. Acceleration speed must be consistent whenever the user moves inside the VE. Any changes in the motion of the user whether in direction or speed such as going forward and backward, slowing down, or stopping and turning while moving or standing still must be in a similar acceleration speed so that the user can feel comfortable with it. Allow user to control and initiate acceleration speed. Users should have an access to initiate and control the acceleration movement speed whenever possible in the VE so that they can avoid unnecessary camera movement such as jerking or shaking that will be uncomfortable for the user. Movement Speed Provide constant velocity in character movement. Users will feel comfortable when they are exploring the VE if the character movement is created based on constant velocity. Creating real-world speed will be comfortable longer for the user to experience. Provide realistic body movement. This refers to adding other body movements for the user to control such as jumping and crouching. Adding different body movements can avoid serious disparity Provide adequate visual cues If the application provides a teleportation navigation function, providing adequate visual cue is necessary to maintain user bearing and preserve their original orientation Field of View Change viewing distance. The user needs to have the ability to change their FOV which is comfortable for them. HMD has the function to change the view distance of FOV which can reduce the level of cybersickness. But, reducing FOV view distance also can reduce the level of immersion. Degree of Control Synchronized camera movement. Users should have the ability to initiate camera movement so that it can reduce the effect of cybersickness. A sudden change in camera movement can cause discomfort to the user to interact with VE. Provide gesture avatar. Having an avatar that foreshadows impending camera movement can help users to anticipate and prepare for visual motion and at the same time improve the user learning experience. (Continued) Table 5. Validated design elements and principle for cybersickness reduction guidelines (Continued) iJIM ‒ Vol. 16, No. 05, 2022 43 Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… Guidelines Design Elements and Principle Description Camera Avoid zooming camera effect. Zooming in or out with the camera can induce cybersickness since it causes unbalanced head and camera movement. So, it is recommended to avoid the cue of zooming camera effect in any IVR application. Avoid using unnecessary camera templates. Using unnecessary camera templates will make the screen or display area crowded which can distract user attention in conducting the activities or grabbing the information in the VE Avoid using the head bobbing effect. Using the head bobbing effect can create a series of inconsistencies and uncomfortable accelerations. Head Tracking Viewpoint Provide responsive user movement display. The display should be a ready and instant response to the user movements at all times without exception Use SDK’s position tracking and head model. Use provided SDK’s can ensure that the virtual camera rotates and move in a manner consistent with head and body movement. Positional Tracking Provide a suitable starting position. The starting user position in the VE is very important in determining the flow of the event in the VE. The flow of the event is based on the user action and movement at the starting point, so placing the right user position, could help users to explore VE properly Limit user area movement/collider component. User movement inside the VE should be limited based on the concept of the application. So, a collider component is required to prevent a user from accessing unnecessary locations or areas in the VE. Provide positional tracking warning signs. Provide users with warnings as they approach (but well before they reach) the edges of the position camera’s tracking volume as well as feedback for how they can re-position themselves to avoid losing track. Persistence Provide persistence movement of 3D model/ character and transition of information. Any movement and transition of information in the VE must be persistent and consistent. The persistence of every character movement and viewing transition can help user to maintain their focus in performing activities in the VE. 5 Significance of study In the nutshell, this study caters one of the important aspects in developing the IVR application. This refers to the proposed design elements and principles of cybersickness reduction guidelines which can be useful and helpful for developer in developing the IVR application and at the same time becoming one of a new learning support tools that will make the learning process become more interactive, effective and enjoyable to be used by the users. On top of that, it is hope that the outcomes of the study such as the result of comparative and proposed design elements and principles of cybersickness reduction guidelines can become guidance for future researchers and at the same time have positive contribution to the body of knowledge. Table 5. Validated design elements and principle for cybersickness reduction guidelines (Continued) 44 http://www.i-jim.org Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… 6 Conclusion In conclusion, the use of VR/IVR as an interactive approach in teaching and learning has a positive impact on student engagement and participation in the learning expe- rience. To some extent, cybersickness is still identified as a problem that affects the quality of the learning experience, especially when interacting with VR (via HMD). In line with that, two objectives were defined in this study to cater to the problem: i) to explore and identify appropriate cybersickness reduction guidelines in developing VR/IVR applications, and ii) to propose the design element and principle of cyber- sickness reduction guidelines. Based on the comparative analysis process, this study has identified and extracted 12 appropriate cybersickness reduction guidelines which beneficial to be adapted in the development process of VR/IVR application. These guidelines have been retrieved from 14 previous studies through a comparative analy- sis process. Having explored and identified the cybersickness reduction guidelines, this study proposed a total of 27 design elements and principles related to those guidelines. Following that, the proposed design elements and principles were evaluated through the expert review process that involved four experts from different expertise and field. From the expert review feedback, the response has been made and the validated design elements and principles of cybersickness reduction guidelines are presented in the pre- vious section. For future work, a comparative analysis for other development design components and elements such as immersion, navigation, and interaction which are applicable for both mobile and tethered platform should be further explored so that the outcome from the analysis can also be beneficial to support the development process of VR and IVR application in both platforms. 7 Acknowledgment This is part of a Ph.D. work. Special thanks to the School of Multimedia Technology and Communication (SMMTC), UUM for the facilities provided. On top of that, we also would like to thank the Ministry of Higher Education (MoHE) for the scholarship support throughout the research study. 8 References [1] Chen, C. J., Lau, S. Y., Chuah, K. M., & Teh, C. S. (2013). Group usability testing of virtual reality-based learning environments: A modified approach. Procedia-Social and Behavioral Sciences, 97: 691–699. https://doi.org/10.1016/j.sbspro.2013.10.289 [2] Roussou, M. (2001). Immersive interactive virtual reality in the museum. Proc. of TiLE (Trends in Leisure Entertainment). [3] Roussou, M. (2000, February). Immersive interactive virtual reality and informal education. In Proceedings of User Interfaces for All: Interactive Learning Environ- ments for Children (pp. 1–9). [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.25.8911 [4] Al Farsi, G., Yusof, A. B. M., Romli, A., Tawafak, R. M., Malik, S. I., Jabbar, J., & Bin Rsuli, M. E. (2021). A review of virtual reality applications in an educational domain. International Journal of Interactive Mobile Technologies, 15(22): 99–110. https://doi. org/10.3991/ijim.v15i22.25003 iJIM ‒ Vol. 16, No. 05, 2022 45 https://doi.org/10.1016/j.sbspro.2013.10.289 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.25.8911 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.25.8911 https://doi.org/10.3991/ijim.v15i22.25003 https://doi.org/10.3991/ijim.v15i22.25003 Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… [5] G. Burdea and P. Coiffet. (2003). Virtual Reality Technology. Presence Teleoperators Virtual Environ. https://doi.org/10.1162/105474603322955950 [6] Lee, E. L., Wong, K. W., & Fung, C. C. (2009). Learning effectiveness in a desktop virtual reality-based learning environment. In 17th International Conference on Computers in Edu- cation, ICCE 2009, 30 November–4 December (pp. 832–839), Hong Kong. [7] Clancy, T. A., Rucklidge, J. J., & Owen, D. (2006). Road-crossing safety in virtual reality: a comparison of adolescents with and without ADHD. Journal of Clinical Child & Adolescent Psychology, 35(2): 203–215. https://doi.org/10.1207/s15374424jccp3502_4 [8] Chuah, K. M., Chen, C. J., & Teh, C. S. (2009, November). ViSTREET: An educational virtual environment for the teaching of road safety skills to school students. In International Visual Informatics Conference (pp. 392–403). Springer, Berlin, Heidelberg. https://doi. org/10.1007/978-3-642-05036-7_37 [9] Feldstein, I., Dietrich, A., Milinkovic, S., & Bengler, K. (2016). A pedestrian simulator for urban crossing scenarios. IFAC-PapersOnLine, 49(19): 239–244. https://doi.org/10.1016/j. ifacol.2016.10.531 [10] Zulkifli, A. N., Mohamed, N. F. F., Qasim, M. M., & Bakar, N. A. A. (2021). Prototyp- ing and usability evaluation of road safety education courseware for primary schools in Malaysia. International Journal of Interactive Mobile Technologies, 15(6): 32–47. https:// doi.org/10.3991/ijim.v15i06.20609 [11] Kusuma, G. A., Wirawan, I., & Arthana, I. (2018). Virtual reality for learning fish types in kindergarten. International Journal of Interactive Mobile Technologies, 12(8): 41–51. https://doi.org/10.3991/ijim.v12i8.9246 [12] Davis, S., Nesbitt, K., & Nalivaiko, E. (2014, December). A systematic review of cybersick- ness. In Proceedings of the 2014 Conference on Interactive Entertainment (pp. 1–9). https:// doi.org/10.1145/2677758.2677780 [13] B. L. Rebenitsch. (2015). Cybersickness in Virtual Reality. 22(1): 46–51. https://doi. org/10.1145/2810054 [14] Porcino, T. M., Clua, E., Trevisan, D., Vasconcelos, C. N., & Valente, L. (2016). Minimizing cybersickness in head mounted display system: Design guidelines and application. In 2017 IEEE 5th International Conference on Serious Games and Applications for Health (SeGAH) (pp. 1–11). https://doi.org/10.1109/SeGAH.2017.7939283 [15] LaViola Jr, J. J. (2000). A discussion of cybersickness in virtual environments. ACM Sigchi Bulletin, 32(1): 47–56. https://doi.org/10.1145/333329.333344 [16] Oculus. (2017). Oculus Best Practices, 3rd ed. Oculus VR LLC. [17] HTC. (2021). VIVE User Guide. https://www.vive.com/eu/support/cosmos/category_howto/ choosing-your-play-area.html (accessed Nov. 21, 2021). [18] E. Games. (2020). Virtual Reality Best Practice in Unreal Engine 4 Documentation. https:// docs.unrealengine.com/4.27/en-US/SharingAndReleasing/XRDevelopment/VR/VRBest- Practices/#vrandsimulationsickness (accessed Nov. 28, 2021). [19] Mousavi, M., Jen, Y. H., & Musa, S. N. B. (2013). A review on cybersickness and usability in virtual environments. In Advanced Engineering Forum, 10: 34–39. Trans Tech Publica- tions Ltd. https://doi.org/10.4028/www.scientific.net/AEF.10.34 [20] Reason, J. T., & Brand, J. J. (1975). Motion sickness. Academic press. [21] Riccio, G. E., & Stoffregen, T. A. (1991). An ecological theory of motion sickness and postural instability. Ecological psychology, 3(3): 195–240. https://doi.org/10.1207/ s15326969eco0303_2 [22] Ibrahim, N., & Ali, N. M. (2018). A conceptual framework for designing virtual heritage envi- ronment for cultural learning. Journal on Computing and Cultural Heritage (JOCCH), 11(2): 1–27. https://doi.org/10.1145/3117801 46 http://www.i-jim.org https://doi.org/10.1162/105474603322955950 https://doi.org/10.1207/s15374424jccp3502_4 https://doi.org/10.1007/978-3-642-05036-7_37 https://doi.org/10.1007/978-3-642-05036-7_37 https://doi.org/10.1016/j.ifacol.2016.10.531 https://doi.org/10.1016/j.ifacol.2016.10.531 https://doi.org/10.3991/ijim.v15i06.20609 https://doi.org/10.3991/ijim.v15i06.20609 https://doi.org/10.3991/ijim.v12i8.9246 https://doi.org/10.1145/2677758.2677780 https://doi.org/10.1145/2677758.2677780 https://doi.org/10.1145/2810054 https://doi.org/10.1145/2810054 https://doi.org/10.1109/SeGAH.2017.7939283 https://doi.org/10.1145/333329.333344 https://www.vive.com/eu/support/cosmos/category_howto/choosing-your-play-area.html https://www.vive.com/eu/support/cosmos/category_howto/choosing-your-play-area.html https://docs.unrealengine.com/4.27/en-US/SharingAndReleasing/XRDevelopment/VR/VRBestPractices/%23vrandsimulationsickness https://docs.unrealengine.com/4.27/en-US/SharingAndReleasing/XRDevelopment/VR/VRBestPractices/%23vrandsimulationsickness https://docs.unrealengine.com/4.27/en-US/SharingAndReleasing/XRDevelopment/VR/VRBestPractices/%23vrandsimulationsickness https://doi.org/10.4028/www.scientific.net/AEF.10.34 https://doi.org/10.1207/s15326969eco0303_2 https://doi.org/10.1207/s15326969eco0303_2 https://doi.org/10.1145/3117801 Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… [23] Szczurowski, K., & Smith, M. (2018, August). “Woodlands”—a virtual reality serious game supporting learning of practical road safety skills. In 2018 IEEE Games, Entertainment, Media Conference (GEM) (pp. 1–9). IEEE. https://doi.org/10.1109/GEM.2018.8516493 [24] Dorado, J. L., & Figueroa, P. A. (2015, March). Methods to reduce cybersickness and enhance presence for in-place navigation techniques. In 2015 IEEE Symposium on 3D User Interfaces (3DUI) (pp. 145–146). IEEE. https://doi.org/10.1109/3DUI.2015.7131742 [25] Tan, C. T., Leong, T. W., Shen, S., Dubravs, C., & Si, C. (2015, October). Explor- ing gameplay experiences on the Oculus Rift. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (pp. 253–263). https://doi. org/10.1145/2793107.2793117 [26] Fedorov, N. (2015). Frame Rate (FPS) vs Refresh Rate (Hz). AvaDirect. https://www. avadirect.com/blog/frame-rate-fps-vs-hz-refresh-rate/ (accessed Feb. 16, 2019). [27] Rebenitsch, L., & Owen, C. (2014, October). Individual variation in susceptibility to cyber- sickness. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (pp. 309–317). https://doi.org/10.1145/2642918.2647394 [28] Sween, R., Deb, S., Carruth, D. W., Waddell, D., & Furuichi, M. (2017). Development of an effective pedestrian simulator for research. In Advances in Human Aspects of Transporta- tion (pp. 183–191). Springer, Cham. https://doi.org/10.1007/978-3-319-41682-3_16 [29] Kemeny, A., George, P., Mérienne, F., & Colombet, F. (2017). New VR navigation tech- niques to reduce cybersickness. Electronic Imaging, 2017(3): 48–53. https://doi.org/10.2352/ ISSN.2470-1173.2017.3.ERVR-097 [30] R. S. Wales. (2017). Virual Reality App to Teach Road Safety to Children. https://www. roadsafetywales.org.uk/news/posts/2017/september/virtual-reality-app-to-teach-road-safe- ty-to-children/?Language=undefined (accessed Jul. 31, 2019). [31] Michalak, S. (2017). Guidelines for immersive virtual reality experiences. White Paper, Jul. [32] Chang, T. R., Kaasinen, E., & Kaipainen, K. (2012, November). Persuasive design in mobile applications for mental well-being: multidisciplinary expert review. In International Confer- ence on Wireless Mobile Communication and Healthcare (pp. 154–162). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37893-5_18 [33] Wolfenstetter, T., Füller, K., Böhm, M., Krcmar, H., & Bründl, S. (2015, October). Towards a requirements traceability reference model for Product Service Systems. In 2015 International Conference on Industrial Engineering and Systems Management (IESM) (pp. 1213–1220). IEEE. https://doi.org/10.1109/IESM.2015.7380307 [34] S. Zulaiha. (2017). A Conceptual Model of Interactive Computer Assisted Learning for Low Achieving Primary School Student, Universiti Utara Malaysia. [35] Nurulnadwan, A. (2015). Conceptual design and development model of assistive courseware for young low vision learners (AC4LV) (Doctoral dissertation, Universiti Utara Malaysia). 9 Authors Nur Sauri Yahaya has a bachelor’s degree in Multimedia at Universiti Utara Malay- sia (UUM) in 2014 and a Master’s in Creative Industries majoring in Interactive and Visual Design at Queensland University of Technology, Australia in 2017. He currently pursuing a Ph.D. (Multimedia) in Universiti Utara Malaysia (UUM). His research inter- est is related to Virtual Reality, Immersive Virtual Reality, Augmented Reality, and Educational Technology. iJIM ‒ Vol. 16, No. 05, 2022 47 https://doi.org/10.1109/GEM.2018.8516493 https://doi.org/10.1109/3DUI.2015.7131742 https://doi.org/10.1145/2793107.2793117 https://doi.org/10.1145/2793107.2793117 https://www.avadirect.com/blog/frame-rate-fps-vs-hz-refresh-rate/ https://www.avadirect.com/blog/frame-rate-fps-vs-hz-refresh-rate/ https://doi.org/10.1145/2642918.2647394 https://doi.org/10.1007/978-3-319-41682-3_16 https://doi.org/10.2352/ISSN.2470-1173.2017.3.ERVR-097 https://doi.org/10.2352/ISSN.2470-1173.2017.3.ERVR-097 https://www.roadsafetywales.org.uk/news/posts/2017/september/virtual-reality-app-to-teach-road-safety-to-children/?Language=undefined https://www.roadsafetywales.org.uk/news/posts/2017/september/virtual-reality-app-to-teach-road-safety-to-children/?Language=undefined https://www.roadsafetywales.org.uk/news/posts/2017/september/virtual-reality-app-to-teach-road-safety-to-children/?Language=undefined https://doi.org/10.1007/978-3-642-37893-5_18 https://doi.org/10.1109/IESM.2015.7380307 Paper—A Comparative Analysis on Cybersickness Reduction Guidelines in VR and IVR Applications… Ariffin Abdul Mutalib is an Associate Professor of Multimedia at the School of Multimedia Technology and Communication, Universiti Utara Malaysia (UUM). He holds a bachelor’s degree in Information Technology at Universiti Utara Malaysia (UUM), a master’s degree in Interactive Multimedia at Heriot-Watt University, Edin- burgh, UK, and a Ph.D. degree in Information Technology at Universiti Utara Malaysia (UUM). His research interest is diverse and focuses on multimedia, interactive design, user-centered design, and user-experience. E-mail: am.ariffin@uum.edu.my Sobihatun Nur Abdul Salam is an Associate Professor and Dean at the School of Multimedia Technology and Communication, Universiti Utara Malaysia (UUM). She received a bachelor’s and master’s degree in Information Technology at Universiti Utara Malaysia (UUM) and a Ph.D. degree in Multimedia at Universiti Sains Malaysia (USM). Her research interest is diverse and focuses on persuasive technology, persuasive multimedia, assistive technology, and gamification. E-mail: sobihatun@uum.edu.my Article submitted 2021-08-19. Resubmitted 2021-12-07. Final acceptance 2021-12-14. Final version published as submitted by the authors. 48 http://www.i-jim.org mailto:am.ariffin@uum.edu.my mailto:sobihatun@uum.edu.my