Frontiers in Computing and Intelligent Systems ISSN: 2832-6024 | Vol. 3, No. 3, 2023 43 Influence of Spatial Perception Ability on Virtual Annotation Response in Teleoperation of Excavator Xiaomeng Li a, Xing Su b School of Civil Engineering and Architecture, Zhejiang University, Hangzhou 310000, China a 22012310@zju.edu.cn, b xsu@zju.edu.cn Abstract: Virtual annotation (VA) for compromising visual limitations during teleoperation can provide critical instructional information, but it remains to be verified whether an operator can respond to VA timely and accurately while operating. Studies have demonstrated a positive relationship between spatial perception ability (SPA) and teleoperating performance. Many studies considered that those with greater SPA have a lower mental workload and more attention on other matters. Therefore, this study sought to examine whether an operator with greater SPA can respond better to VA. Quantitative results from the SPA test questionnaire and the teleoperated excavator experiment showed a significant inverse relationship between SPA and the response time. It is expected that the results could improve the effectiveness of selecting teleoperators and optimize prior management. Keywords: Teleoperation; Virtual Annotations; Spatial Perception Ability. 1. Introduction Teleoperation of excavator can isolate operators from the hazardous work environment to guarantee their safety and has been studied for some extreme scenarios, such as construction rescue in earthquake secondary disaster environment. However, the research on teleoperation is mostly in the stage of theoretical development. Existing researches mainly focus on the design of operating system and hardware equipment, such as setting sensors or cameras that can obtain field image information and designing data contact interfaces between the field and remote operation terminal [1, 2].The availability and productivity of teleoperation are usually lower than that of traditional operations [3].The main reason is that the operators, as the important information receiver, decision and action party in the teleoperating system, is limited in the ability to receive information and thus perceive the situation on the site. In teleoperating systems, operators usually use images in the visual interface where 3D information is displayed on 2D screens to understand the remote situation. In the indirect perception mode, the operators’ perspective and field of view are limited and the situational awareness is insufficient, which is easy to compress the real perception and underestimate the egocentric distance, which greatly affects the effectiveness of teleoperation. In this regard, some studies have proposed to install multiple cameras or draw 3D maps of the building site to reproduce the remote environmental information from different angles. However, these methods bring in too much visual information, and the operator often has difficulty integrating them and feels overloaded or tired with mental workload. Virtual annotation (VA) is an effective solution because it can provide simple and direct visual information prompt in a short time to supplement the loss of visual perception. It doesn’t take much mental workload for an operator to understand the warning or task. For example, highlighting the edge of a dangerous obstacle or setting up a radar map in the corner of the screen can make it easy for the operator to find the location of the obstacle [4]. VA is a form of information assistance that presents key prompts (such as obstacles, emergencies, critical spacing, etc.) to the operator in a user-friendly and easy-to-understand manner. At present, in the field of construction, although there are studies trying to use VA as a visual aid, most of them tend to focus on visual presentation technology and detection sensing technology, without considering whether VA really plays its role. The effectiveness of VA application can be demonstrated only when operators can make corresponding responses to different VAs timely and accurately. In the case of relatively mature VA technology, the remaining variable, the quality and ability of the “operator” displayed in teleoperation has a great influence on operation performance and response to VA. It has been proved that operators have an important influence on the objectives realization in the teleoperation system. They need to acquire and process information about the working environment and surrounding conditions to make operational plans, decisions and actions. Although excavator operators receive pre-operation training, the training is mainly to understand the operational requirements. The actual operational proficiency may vary from individual to individual, and one of the major differences is spatial perception ability (SPA) [5]. As an important component of human intelligence, SPA is one of the most studied and valuable factors in human cognitive function. A large number of studies have shown that the performance of mechanical teleoperation is related to SPA. Operators with greater SPA can complete operational tasks faster and have higher productivity, because they can form the overall cognitive map faster, eliminate the differences between individual spatial representations through learning effect, quickly understand the work scene and task requirements, have a lower mental workload during work [6], effectively realize hand-eye coordination, and thus perform better in teleoperation. As mentioned earlier, the introduction of VA reduces some mental workload. However, VA, as a newly added element, increases the cognitive needs to distinguish different types of VAs and reflect corresponding operations than the previous pure operational tasks. Moreover, VA in the teleoperation of excavator is mainly used to remind obstacles and emergencies. Existing studies have investigated the relationship between 44 SPA and teleoperation performance, without considering external visual cues. Therefore, on the one hand, this paper aims to explore the teleoperational performance of excavators with different SPA, which has not been studied; on the other hand, it aims to explore operators’ reactions with different SPA to VA outside excavators’ operation, in order to provide guidance for operators’ selection and training. 2. Related Work 2.1. Virtual Annotation (VA) At present, various types of VA have been successfully applied to the fields of navigation planning, surgical training, maintenance and inspection, and construction machinery operation. VA is applied to provide additional explanatory information to the actual objects in media interfaces such as AR devices and computer screens to facilitate user understanding. In the field of navigation planning, Zetzsche et al. used text and graphics to annotate traffic conditions and route information in the simulation of aircraft operation, so as to timely guide the flight direction and reduce the probability of accidents [7]. Bolton et al. designed a navigation system that used anchored annotations to highlight landmarks. Due to the timely observation of annotations, the response time for subjects to see and respond to landmarks was reduced by 43.1% and the response success rate increased by 26.2% [8]. Guven and Feiner added VA to the AR system of the building so that mobile users can receive timely information about their location on the device while touring the building [9]. In the field of surgical training, VA was generally used to highlight the target area, annotate the key area, guide the surgical route and process. Andersen et al. guided the trainees to hold the scalpel, cut and suture the wound according to the indicated route through graphic markers, and could learn the operation repeatedly, effectively improving the training effect [10]. Leo et al. believed that VA can also be directly set on the heart model. They had built CardiacAR software, which allowed users to click any position on the heart model and attach annotations, so as to facilitate surgeons to annotate different parts of the heart model in real time and review them as required [11]. In the field of maintenance and inspection, VA can help non-specialists locate faults and follow instructions to communicate with remote repairmen. Gauglitz et al had developed a visual space sharing enhancement system where users can communicate with each other through AR spatial annotation, and remote maintenance of devices can be completed with the support of the system [12]. Leutert et al proposed an AR system in which users can add VA to highlight faulty mechanical parts to facilitate remote maintenance communication of equipment [13]. In the field of construction machinery operation, there were few researches related to VA. One part of these researches was to use VA for safety monitoring and warning, such as the VA developed by Fang et al. for target positioning and safety early warning in the visual interface of auxiliary tower crane operation [14]. Chen et al. also proposed a VA to display object distance and warn dangerous areas in a visualization platform for construction activity monitoring [15].The another part was to use VA to assist the teleoperation of construction machinery. Among them, the research theory and experiment of VA was relatively complete in the teleoperation of excavator proposed by Hong et al. VA helped operators to understand the field situation by highlighting the edge of obstacles or setting radar map on the screen to obtain the relative position of excavator and surrounding objects. In addition, VA often appeared in augmented reality (AR). At present, AR is mainly used in teleoperation to add annotations to virtual simulation scenes to indicate the operation of machinery, so as to achieve improved operation success rate [16]. Although VA has been applied in most fields, it is still in the exploratory stage in the field of construction machinery teleoperation. Exploring the effectiveness of VA application in this field will improve the working efficiency of teleoperation from the point of view of improving the design of human-computer interaction interface of teleoperation. 2.2. Spatial Perception Ability (SPA) There is no clear standard for the definition of spatial perception ability (SPA). DU et al. believe that spatial perception is the process of determining an individual’s location in space, creating information about space in mind, identifying and understanding space, finding places, finding routes and describing places. Menaca -Brandan et al. consider that SPA refers to the ability to generate, visualize, remember, and transform visual information such as pictures, maps, 3D images, etc. [17]. Hegarty believes that spatial perception is an important aspect of human perception, which is essentially a process related to the interpretation of the incoming data of personal visual perception, including images, objects, maps, and scenes of specific situations in the environment [18]. Based on the definitions of these different dimensions, scholars generally consider that SPA is an important dimension of human intelligence, and humans with greater SPA have more advantages in understanding, remembering, inferring and expressing spatial relationships. Although SPA is the inherent ability of human beings, it is not unchangeable. SPA is not a whole trait, but contains several sub-abilities. As people grow up, these sub-abilities will change, so will the SPA. Ekstrome et al. divided SPA into spatial orientation (SO) and spatial visualization (SV) abilities [19]. Pellegrino and Long et al. believed that SPA included spatial visualization (SV) and spatial relation (SR) [20, 21]. Other scholars [22, 23] proposed the three classification models of SPA: SO, SV and SR. SO, refers to the ability of people to obtain and maintain a specific perspective through the relative relationship between other objects in space, such as the position of something relative to other objects in the environment. SV refers to the ability of people to imagine the rotation, translation, folding or expansion of plane figures. SR, also known as mental rotation, is to imagine the appearance of an object after three- dimensional arbitrary flipping. It follows that SPA is related to the cognitive abilities involved in understanding 3D forms, understanding the position of objects in space, and transforming positions with respect to objects and other viewpoints. Many studies have shown that people with greater SPA have more advantages in STEM fields including science, technology, engineering and mathematics. Potential advantages of greater SPA are also found in fields that require three-dimensional spatial imagination. For example, Dror et al. argued that pilot selection should be based on SPA, and try to select “natural” (greater SPA) pilots. The reason is that mental rotation, abstract reasoning and spatial orientation are important factors valued in pilot selection tests, and these 45 abilities correspond to the above-mentioned three classification model [24]. Menaca -Brandan et al. found that astronauts with greater SO and SR abilities spent significantly shorter operating time and observation time in the experiment through simulation experiments of operating extra vehicle robotic arms, indicating that there is a significant positive correlation between SPA and operational performance. Liu et al. proved through experiments that SPA-related test results could be used to predict astronauts’ performance in operating extravehicular robotic arms, suggesting that astronauts with low SPA scores could be trained in advance. In addition, since astronauts with better space skills are more likely to perform well, their work can be arranged more flexibly within a shorter time window. Lathan et al. showed that operators’ SPA was different among individuals, which could be reflected in teleoperation performance. The greater the SPA was, the better the teleoperation performance would be, mainly because the strength of SPA would affect the integration speed and accuracy of operators’ spatial information from different perspectives. These studies have proved that SPA is closely related to the machinery teleoperation performance. However, current teleoperation studies related to SPA are limited to the mechanical operation itself and do not further consider whether operators with different SPA can effectively respond to VA. In this paper, the relationship between SPA and response to VA will be explored experimentally, and the characteristics of “operator” that affect the overall performance of teleoperation will be discussed, in order to support the early selection of operators and the ability orientation in process training. 3. Experiment This study adopted the simulation experiment method to explore the relationship between SPA and the response to VA, built the excavator teleoperation system on the Unity3D platform, recruited subjects to complete the excavator teleoperation experiment, and obtained experimental performance data. Paper test questionnaire was used to obtain the scores of each subject’s SPA and its sub-items, and statistical analysis method was used to compare and analyze the two types of data. 3.1. SPA Test Questionnaire Design Since the screen image watched by the subjects during the operation is from a self-centered perspective, there is no need to convert the perspective to observe the external view centered on the external world. The way to obtain the remote situation by integrating the center and external view has been realized by VA which is easy to understand. Therefore, SO ability is not involved in this experiment. And because in the binary method of SPA, SV and SR ability can be uniformly classified into SV ability, so this experiment adopts SV and SR two sub-abilities to test SPA. As shown in Table 1, the two measurement methods of SV mainly refer to the way people imagine in their minds the flat figure after folding and expanding, and can integrate several figures into a new figure by horizontal rotation and translation; the four test methods of SR mainly refer to the pattern that people imagine in their minds that flat or three-dimensional shapes are presented after a series of horizontal rotations and arbitrary flips. The final set consisted of 21 questions, and subjects were required to complete the test within 15 minutes, and the number of correct answers was used as the test score. Table 1. Composition of SPA test question set Sub-items Test Question Question Type Number SV Paper Folding Single Choice 6 Form Board Single Choice 3 SR Card Rotation Single Choice 3 Cube Comparison Single Choice 3 PSVT: R Single Choice 3 andenberg MR Multiple Choice 3 3.2. Teleoperation Experiment Design In this study, the experimental system was developed in C#. The models of excavator and balls were downloaded in Github and imported into the Unity3D engine. Fig. 1 shows the scene in which the excavator work part operation is completed in the experiment, which only involves the transfer of the ball through the operation of the main body rotation, bucket, stick, and boom, so no other modifications are required for the experimental scene. Visual Studio was used to write scripts to control the behavior of objects in the experimental system, position the camera, and determine the participant-centered perspective, that is, the driver’s perspective from the cab shown on the screen of Fig. 2 to complete the interactive tasks in the experimental system. Fig 1. Experimental scene in Unity3D Fig. 2 shows the experimental scenario and equipment. The teleoperation platform is deployed on a computer with 3.70 GHz Intel(R) Core(TM), 64G RAM, and NVIDIA GeForce RTX 2080 Ti with 11,048 MB VRAM. The screen in front of the subjects was used to display the teleoperation scenario, with a game joystick in each left and right hand and a keyboard placed in the middle for the necessary key control. Fig 2. Experimental equipment: desktop display, game joystick 46 Two types of VA, ring and cross (refer to Fig. 3), were used to represent random obstacles or contingencies in the construction environment. Subjects need to respond promptly and accurately to them during teleoperation. The ring-shaped VA requires the operator to push the honk button while excavating, and the cross-shaped VA requires the operator to cease operation until the VA vanishes. VA appears randomly anywhere on the screen. Fig 3. Diagram of the appearance of ring and cross-shaped VA in teleoperation 3.3. Participants Participants were recruited from Zhejiang University student population. A total of 30 students participated, 15 were male and 15 were female. Their mean ages were 24.03 years with corresponding standard deviations of 2.03, respectively. All participants have no construction equipment operation experience. 3.4. Experiment Procedure First, subjects were required to fill out the pre-task questionnaire to provide information about gender, age, and previous 3D gaming experience. Then, they should watch a short video introducing excavator operation, and they were given five minutes to familiarize the operation. Next, they should watch another short video introducing the reaction to VA, and two minutes were given to practice it. The last was the ten-minute formal test, in which two types of VA appeared randomly anywhere on the screen, and the number of balls transferred and the number of correct VA responses were displayed at the top left of the screen. The backend will also record the right or wrong and time of each response to VA. The SPA questionnaire was administered on another day, and participants were given 15 minutes to complete a paper questionnaire. 3.5. Evaluation Indicators and Research Hypothesis The average response time and response accuracy excluding non-response, the average response time counting only the correct response and the number of balls transferred were the four indicators of experimental performance, which were represented by Tnun, Rnun, Tcorr and Ntran respectively. The research hypothesis was based on the evaluation indicators: SPA is significantly negatively correlated with values of Tnun and Tcorr, and positively correlated with values of Rnun and Ntran. 4. Results 4.1. Descriptive Statistics Descriptive statistics are performed on the scores of SPA and its sub-abilities, SV and SR, as shown in Table 2, and show that SPA scores range from 7 to 18 (mean 12.2, standard deviation 2.5141), SV scores range from 0 to 5 (mean 2.73, standard deviation 1.4610), and SR scores range from 5 to 13 (mean 9.47, standard deviation 1.6914). The total score of the questionnaire was 21 points, and there were 30 valid questionnaires. The values of the SPA, SV and SR conform to a normal distribution. Table 2. Descriptive statistics of SPA, SV and SR SPA items N Min Max Mean Std. SPA 30 7 18 12.20 2.5141 SV 30 0 5 2.73 1.4610 SR 30 5 13 9.47 1.6914 According to the meaning, calculation method and background data of the indicators in 3.5, the values of the corresponding evaluation indicators are calculated, and the descriptive statistical results are shown in Table 3. Table 3 indicates that the overall value of the Tcorr indicator is larger than the value of the Tnun indicator, which is in line with the reality that “it usually takes more time to complete the correct response”. Table 3. Descriptive statistics of evaluation indicators Indicators Min Max Mean Std. Tnun 0.589 1.339 0.868 0.1666 Tcorr 0.640 1.381 0.935 0.1729 Rnun 0.558 0.925 0.795 0.095 Ntran 158 679 372 129 In addition to these four indicators, this paper also makes classification and description statistics on one of the potential influencing variables, the type of VA, which is due to the difference in the response requirements of the cross-shaped and ring-shaped VAs to the participants, which may lead to the impact of the analysis conclusions when these two types of VAs are put together for data analysis. Thus, Table 4 shows the descriptive statistical results of two VAs calculated from the background data under the three indicators. The results indicate that the values of Tnun and Tcorr indicators in the cross-shaped VA response are smaller, and the values of the Rnun indicator are larger, which reflects the characteristics of more time-consuming and difficult response of ring-shaped VA. Table 4. Descriptive statistics of evaluation indicators under different VAs Indicators VAs Min Max Mean Std. Tnun cross 0.545 1.291 0.8129 0.1742 ring 0.612 1.511 0.9208 0.1982 Tcorr cross 0.609 1.397 0.9179 0.1883 ring 0.695 1.462 0.9810 0.2080 Rnun cross 0.833 1.000 0.9740 0.0397 ring 0.188 0.875 0.6141 0.1913 Before discussing the relationship between SPA and the four indicators of operational performance, it is necessary to understand the correlation between SPA and its sub-items, as shown in Table 5. Table 5 shows that there is a strong correlation between SPA and SR (r=0.745, p=0.000), the correlation between SPA and SV is weaker (r=0.473, p=0.008), and the correlation between SV and SR is weaker and less significant (r=0.268, p=0.151). 47 Table 5. Correlation between SPA and its sub-items Test items SPA SV SR SPA Pearson correlation coefficient 1 0.473** 0.745** Sig. (Two- tailed) 0.008 0.000 SV Pearson correlation coefficient 0.473** 1 0.268 Sig. (Two- tailed) 0.008 0.151 SR Pearson correlation coefficient 0.745** 0.268 1 Sig. (Two- tailed) 0.000 0.151 Since the correlation between SPA and its sub-items will cause independent variable multicollinearity problem, it is not suitable to use linear regression method to analyze its impact on operation performance evaluation index. Correlation analysis will be used to analyze the relationship between SPA items and the corresponding dependent variable indexes, which has been successfully applied in the research of Menchacha-Brandan et al. 4.2. Correlation Analysis 4.2.1. Average Response Time The correlation between Tnun and Tcorr indexes and SPA and its sub-items was obtained by using the SPSS bivariate correlation analysis function. All values conform to a normal distribution. SPA, SV and SR all showed a negative correlation with Tnun and Tcorr indexes, but the relationship is not significant (p>0.05), see Table 6. Table 6. Correlation analysis of SPA items with dependent variables Tnun and Tcorr indicators Test items SPA SV SR Tnun Pearson correlation coefficient -0.144 -0.160 -0.058 Sig.(Two- tailed) 0.447 0.397 0.760 Tcorr Pearson correlation coefficient -0.172 -0.161 -0.124 Sig.(Two- tailed) 0.363 0.394 0.514 Table 7. Correlation analysis of SPA items with Tnun and Tcorr indicators corresponding to two VAs SPA items Indicators VAs Pearson correlation coefficient Sig. (Two-tailed) SPA Tnun cross -0.077 0.690 ring -0.411* 0.024 Tcorr cross -0.077 0.680 ring -0.402* 0.028 SV Tnun cross -0.057 0.770 ring -0.360 0.053 Tcorr cross -0.080 0.681 ring -0.340 0.067 SR Tnun cross 0.001 0.993 ring -0.394* 0.033 Tcorr cross -0.008 0.971 ring -0.400* 0.028 In this regard, the situation of different response difficulties of cross-shaped and ring-shaped VAs described in 4.1 can be linked, and two kinds of VAs can be separated from the SPA items for correlation analysis, and the results are shown in the Table 7 and Fig. 4. When the two types of VAs are discussed separately, the Tnun and Tcorr indicators corresponding to the ring-shaped VA have a significant negative correlation with the SPA and SR items, but the Tnun and Tcorr indicators corresponding to the cross-shaped VA have no significant negative correlation with the three index items of SPA. Fig 4. Correlation analysis of SPA items with Tnun and Tcorr indicators corresponding to two VAs 4.2.2. Response Accuracy Table 8 shows that SPA, SV and SR are not significantly correlated with Rnun index (p>0.05), but there is a positive correlation trend overall. Table 9 shows that when the two types of VAs are discussed separately, the Rnun indicator value still has no significant positive correlation with the SPA items. Table 8. Correlation analysis of SPA items with Rnun Text items SPA SV SR Rnun Pearson correlation coefficient 0.284 0.129 0.222 Sig. (Two-tailed) 0.128 0.495 0.238 Table 9. Correlation analysis of SPA items with Rnun corresponding to two VAs SPA items Indicator VAs Pearson correlation coefficient Sig.(Two- tailed) SPA Rnun cross 0.086 0.651 ring 0.182 0.333 SV Rnun cross -0.201 0.311 ring 0.153 0.441 SR Rnun cross 0.038 0.840 ring 0.141 0.453 48 4.2.3. The Number of Balls Transferred The Ntran indicator measures the number of balls transferred by participants in the same amount of time, representing the productivity of excavator teleoperation. The results of correlation analysis (refer to Table 10 and Fig. 5) showed that SPA and SR had a significant positive correlation with the Ntran index (p=0.008, p=0.004), which verified the hypothesis that the greater the spatial perception ability, the higher the mechanical operation performance. SV value has a weak correlation with the Ntran index, and the relationship in not significant. Table 10. Correlation analysis of SPA items with Ntran Text items SPA SV SR Ntran Pearson correlation coefficient 0.477** 0.022 0.513** Sig. (Two-tailed) 0.008 0.920 0.004 Fig 5. Correlation analysis of SPA items with Ntran 5. Data Analysis and Interpretation The positive relationship between SPA and SR and operational performance (the number of balls transferred in this study) has been demonstrated by existing studies. The results once again verified the positive relationship between spatial perception ability and teleoperation performance, and can explain the validity of the data of this experiment to some extent. The inverse relationship between SPA and SR and reaction time stems from the ability to perceive the global picture and the mental workload associated with spatial perception ability. Mental workload is a measure of how well a person’s information processing system is used when the system is working. The aforementioned studies have demonstrated that a person with better spatial perception ability has lower mental workload, so it is reasonable to assume that he has a greater global observation ability and enough attention to perceive things beyond the operation itself.This was demonstrated in this experiment by the fact that those with better SPA and SR had shorter mean reaction times to the ring- shaped VA. However, for the response of cross-shaped VA, reaction times did not differ significantly. In the feedback comments, most subjects mentioned that the cross is often encountered in real life and usually represents the meaning of “stop”, so they can respond correctly in time without thinking much, and then there is no significant difference in this index between different spatial perception ability. As can be seen from Table 4, the mean response time corresponding to cross- shaped VA is shorter than that of ring-shaped VA, and the mean response accuracy is higher than that of ring-shaped VA. Therefore, two conclusions can be drawn: (1) When the cognitive/mental workload and hand-eye coordination requirements of a certain VA are very low, there is no significant difference in the response of operators with different spatial perception ability to it, and they can all make effective response to it in time, and the response rate is high; (2) With the increase of cognitive/mental workload and hand- eye coordination requirements attached to VA, the response accuracy will decrease, and the response time will increase. Cognitive workload is a similar concept to mental workload. Mental workload is mainly based on the assumption that human cognitive ability is limited, the main motivation is to quantify the mental cost of performing tasks, in order to predict the performance of the operator and the system, and the ultimate goal is to develop work content that matches the level of mental workload. Cognitive workload is mainly based on the assumption that human’s working memory ability is limited, and more emphasis is placed on the design of system form. The ultimate goal is to design appropriate training environment or instruction, without imposing unnecessary extra cognitive load and wasting cognitive resources. The cross-shaped VA design itself had a lower cognitive load, so there was no significant difference in response.Since teleoperators are often in a highly stressful work situation, any slight increase in difficulty and cognitive load would represent a dramatic escalation of difficulty for them. Therefore, VA in teleoperating systems should be associated with a low cognitive load. For example, the form and color of VA can be selected to match the common response scenarios in real life. For the response accuracy, all the results are not significant, which also may be related to the simplicity of the response requirement of the ring and cross. The 2-minute practice before the formal experiment and the learning process during the formal experiment were sufficient for the subjects to remember the operation, which resulted in no significant difference in the correctness index between different spatial perception ability. The assessment indicator associated with response accuracy was Tcorr. Tcorr and Tnun were similar to the results of the spatial perception ability analysis.The reason for using this indicator is that the goal of teleoperation is to respond quickly and accurately to the VA. Some operators may respond quickly but operate incorrectly; Some operators may respond correctly but operate too slowly, both of which are detrimental to the safe and efficient completion of teleoperation tasks. Again, it was logical that there was no significant positive correlation between Rnun and spatial perception ability. Based on the “multiple resources theory”, subjects with greater spatial perception ability can understand the task environment faster, master operation skills, and have relatively few mental resources for the main task of transferring small balls, so there will be more mental resources and attention to pay attention to and identify different VAs, mobilize mental resources to think about the requirements of different VAs, and quickly complete the response operation through hand-eye coordination. Therefore, Tcorr with both time and accuracy is the most representative index for the purpose of this experiment. Although the Rnun index has nothing to do with the strength of spatial perception ability in this experiment, due to the lack of relevant studies, in order to ensure the accuracy of the conclusions, experiments can be added in the future, the difficulty and type of VA operations can be increased in the experiment, or the obstacles that may be encountered in 49 the actual work scene can be represented in a variety of different VA forms, so as to re-explore the relationship between spatial perception ability and response accuracy in various situations, and then support the design of VA in remote operating systems. Based on the conclusion that “the greater the spatial perception ability, the significantly shorter the response time of the participant to respond to the VA”, we can select construction machinery teleoperators by examining the SPA of personnel in advance. The examination of SPA still adopts the method of test question set, and the test questions corresponding to SV, SR, and SO abilities are selected according to the work content, and the relevant test questions can also be changed according to the actual situation. In the case of a gradual shortage of labor in the construction industry, the way to effectively train operators in a timely manner to improve SPA is more realistic, which is also part of the increasing proportion of excavator teleoperation research on operators, that is, using VR-based training methods, which can provide operators with an immersive and interactive virtual environment to help them better understand and master operational skills. Researches have proved that SPA can be gradually improved through continuous spatial manipulation exercises. In addition, training methods based on VR technology can also be used for operator selection. By observing how operators perform in a virtual environment, their spatial awareness and operational skills can be better assessed, providing a reference for selecting the appropriate operator. 6. Conclusion The overarching goal of this study was to investigate the responses to VA of operators with different SPA levels during excavator teleoperation. The study collected quantitative experimental data based on the excavator simulator operating platform and SPA test question sets, which were subjected to correlation analysis to test the hypotheses. The analyzed data showed that those with greater SPA had shorter reaction time and more balls transferred, but there was no significant difference in the correctness of the VA response among the different SPA. The result can optimize the prior management of excavator teleoperations to some extent, by incorporating the SPA test into the assessment criteria and intensive training. For the response accuracy, we suggest future investigations with more subjects, better experimental design and test equipment. Testing on different types of construction equipment will also be helpful. References [1] Ha Q, Yen L, Balaguer C. Robotic autonomous systems for earthmoving in military applications [J]. Automation in Construction, 2019, 107: 102934. [2] Zhou T, Zhu Q, Du J. Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction [J]. Advanced Engineering Informatics, 2020, 46: 101170. [3] Nagano H, Takenouchi H, Cao N, et al. Tactile feedback system of high-frequency vibration signals for supporting delicate teleoperation of construction robots [J]. Advanced Robotics, 2020, 34(11): 730-743. [4] Hong Z, Zhang Q, Su X, et al. Effect of virtual annotation on performance of construction equipment teleoperation under adverse visual conditions [J]. Automation in Construction, 2020, 118: 103296. [5] Su X, Talmaki S, Cai H, et al. Uncertainty-aware visualization and proximity monitoring in urban excavation: a geospatial augmented reality approach [J]. Visualization in engineering, 2013, 1(1): 1-13. [6] Liu A M, Oman C M, Galvan R, et al. Predicting space telerobotic operator training performance from human spatial ability assessment [J]. Acta Astronautica, 2013, 92(1): 38-47. [7] Zetzsche C, Galbraith C, Wolter J, et al. Navigation based on a sensorimotor representation: a virtual reality study[C]. Human Vision and Electronic Imaging XII. SPIE, 2007, 6492: 526-539. [8] Bolton A, Burnett G, Large D R. An investigation of augmented reality presentations of landmark-based navigation using a head-up display[C]. Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 2015: 56-63. [9] Guven S, Feiner S. Visualizing and navigating complex situated hypermedia in augmented and virtual reality[C]. 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, 2006: 155-158. [10] Andersen D, Popescu V, Cabrera M E, et al. Virtual annotations of the surgical field through an augmented reality transparent display [J]. The Visual Computer, 2016, 32: 1481-1498. [11] Leo J, Zhou Z, Yang H, et al. Interactive Cardiovascular Surgical Planning via Augmented Reality[C]. Asian CHI Symposium 2021. 2021: 132-135. [12] Gauglitz S, Nuernberger B, Turk M, et al. World-stabilized annotations and virtual scene navigation for remote collaboration [C]. Proceedings of the 27th annual ACM symposium on User interface software and technology. 2014: 449-459. [13] Leutert F, Schilling K. Projector-based augmented reality for telemaintenance support [J]. IFAC-PapersOnLine, 2018, 51 (11): 502-507. [14] Fang Y, Cho Y K, Chen J. A framework for real-time pro- active safety assistance for mobile crane lifting operations [J]. Automation in Construction, 2016, 72: 367-379. [15] Cheng T, Teizer J. Real-time resource location data collection and visualization technology for construction safety and activity monitoring applications [J]. Automation in construction, 2013, 34: 3-15. [16] Bolton A, Burnett G, Large D R. An investigation of augmented reality presentations of landmark-based navigation using a head-up display[C]. Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 2015: 56-63. [17] Menchaca-Brandan M A, Liu A M, Oman C M, et al. Influence of perspective-taking and mental rotation abilities in space teleoperation[C]. Proceedings of the ACM/IEEE International Conference on Human-robot interaction. 2007: 271-278. [18] Hegarty M. Components of spatial intelligence [M]. Psychology of learning and motivation. Elsevier. 2010: 265- 297. [19] Ekstrome R, French J, Harman H, et al. Manual for kit of factor referenced cognitive tests [J]. Princeton, New Jersey: Educational Testing Service, 1976. [20] Long L O, Gomer J A, Wong J T, et al. Visual spatial abilities in uninhabited ground vehicle task performance during teleoperation and direct line of sight [J]. Presence: Teleoperators and Virtual Environments, 2011, 20(5): 466-479. [21] Pellegrino J W, Alderton D L, Shute V J. Understanding spatial ability [J]. Educational psychologist, 1984, 19(4): 239-253. 50 [22] Kozhevnikov M, Hegarty M. A dissociation between object manipulation spatial ability and spatial orientation ability [J]. Memory & cognition, 2001, 29: 745-756. [23] Hegarty M, Waller D. A dissociation between mental rotation and perspective-taking spatial abilities [J]. Intelligence, 2004, 32(2): 175-191. [24] Dror I E, Kosslyn S M, Waag W L. Visual-spatial abilities of pilots [J]. Journal of Applied Psychology, 1993, 78(5): 763.