Australasian Journal of Educational Technology, 2017, 33(6). 11 Mobile augmented communication for remote collaboration in a physical work context Jana Pejoska- Laajola, Sanna Reponen, Marjo Virnes, Teemu Leinonen Aalto University Informal learning in a physical work context requires communication and collaboration that build on a common ground and an active awareness of a situation. We explored whether mobile video conversations augmented with on-screen drawing features were beneficial for improving communication and remote collaboration practices in the construction and facility maintenance services sectors. We used field studies in actual work contexts to map how participants solve physical tasks with remote help powered by augmented video calls, and examined how the drawing feature was used in these contexts. The research data were collected from interviews and job shadowing. The results suggest that augmented video calls enhance remote collaboration by allowing workers to point at task objects and locations, thus potentially improving informal workplace learning. Introduction Construction and facility maintenance are trades in which even conservative and traditional working and learning environments are complex and changing. In these fields, both working and learning take place in physical environments where tasks, co-workers and work contexts change; learning and knowing are tacit; and learning happens on the job and is infrequently discussed (e.g., Eraut, 2007; Pink, Lingard, & Harley, 2016). In such working and learning environments, trade workers continuously seek new solutions and practices to facilitate and improve work safety, well-being, training, quality control, and the transfer of innovations from one site to another. Technologies offer new solutions to known issues, but they also require new practices and changes to current processes, which in turn require new skills, competences, and learning. From early visions and experiments, this research field has developed to the stage where it is possible to design working prototypes and study them in real work environments. Leskinen (2008) reports that in mobile technology in the Finnish construction industry, mobile applications with camera, RFID, GPS, GPRS, and WLAN technologies can not only support, but also replace, paperwork for recurring processes and various types of monitoring, including everything from material, equipment, and quality monitoring to personnel, inspections, and work permit controls. In this study, we explored possibilities for augmented reality (AR) to support informal learning in the construction, facilities, and maintenance services industries. Following an inquiry into the history and state of the art of the industry, we designed and developed a mobile app, Social Augmented Reality (SoAR), that combines video calls with a drawing feature to allow workers to ask questions and provide guidance in work situations. We wanted to study how this kind of AR app could change communication and collaboration practices at work, help people in informal learning situations. The iterative development of the AR app was tested in situ in three field studies: two in Finland and one in Germany. We framed the main research question as follows: How can augmented video calls change communication and collaboration practices for informal learning at work? This paper reports the findings from the pilot field studies. We begin by presenting the related research regarding two specific features of the AR communication technology: (1) a shared view, and (2) pointing and drawing. These features offer ways to improve communication and remote collaboration in physical work sites. We then present the SoAR tool that we designed, developed, and used on the selected sites. Next we present the research design. Finally, we present the findings of the current communication practices enhanced through SoAR, users’ expectations for SoAR during field studies, and the advantages and challenges related to the implementation of SoAR in the construction and facility maintenance industries. Australasian Journal of Educational Technology, 2017, 33(6). 12 Augmented communication for remote collaboration In the fields of construction and facility maintenance, a solid understanding of the physical environment is essential for comparing this environment to visual representations of the models that embody a given space (e.g., machines, building objects, and instructions). The role of context in interaction extends further than the basic view of spatial and temporal context that lies at the heart of the ubiquitous computing vision, toward an embodied experience that blends the digital and real (Dourish, 2001). The tools designed for this environment must support access to information on-site and combine real-world and digital-world learning resources (Hwang, Shi, & Chu, 2013). For these reasons, AR and head-mounted displays (HMDs) have gained significant popularity among organisations in the construction and facility maintenance fields, which have frequently been early adopters of such technologies. AR applications support the simultaneous viewing of the physical context and overlaid models or other related information. Depending on the need of a specific project or context, they can be used when needed to track changes in the processes they are inspecting and to notify users about any updates (Zollmann et al., 2014). Using an AR tool in, for example, assembly work can significantly improve the work by reducing errors and allowing the task to be completed faster (Baird & Barfield, 1999). Remote collaboration requires greater attention to immediate physical work situations. Studies of ubiquitous technologies, such as AR technologies (Peng, Su, Chou, & Tsai., 2009), have found that workers who use computing technologies are able to learn the right thing at the right time and make the most use of their immediate situations. These technologies can enhance alertness, awareness of experiences, and reflective processing of the situation within an action. AR may also support the practice of collaborative operation in various locations and provide clues for information retrieval. Furthermore, AR is a potentially suitable approach for strengthening context-related activities by enabling virtual interactions and enhancing relevant information about objects, events and places of interest. Therefore, a viable AR design should satisfy the need to build a professional identity and knowledge pool while enhancing communication and help-seeking in situations that require immediate attention (Pejoska, Bauters, Purma, & Leinonen, 2016). Collaboration over a shared view In a world powered by constant access to the internet, remote collaboration has become a practice without which it would be difficult to imagine working life. Basic text-based services, such as email, chat, and instant messaging, help erase the line of proximity, and when these services are not sufficient, file sharing, screen sharing, and video conferencing offer additional support. In the construction and facility maintenance sectors, communication and collaboration among stakeholders and subcontractors happens regularly, even when some of the communicators are not physically present. Such geographically dispersed communication is often the case when certain events or circumstances need to be visually reproduced in order to be shared and assessed by all actors. The sharing of this visual information is crucial because it creates a common ground for collaboration, serves as a basis for situation mapping, supports the identification of issues, and makes assumptions and beliefs visible. It is also necessity to build situation awareness, which is critical for helping collaborating actors create and maintain a shared understanding (Baker, Hansen, Joiner, & Traum,1999; Clark & Wilkes-Gibbs, 1986). To achieve good situational awareness, make good decisions, and secure positive results, workers must have a solid understanding of both the task at hand and the relevant environment (Endsley, 1995). In supporting the flow of visual information among working groups in construction and maintenance, studies have shown that views of the objects and surroundings relevant to a given project or environment are more useful than the more common facial view in video conferencing tools (Gergle, Kraut, & Fussell, 2013; Licoppe & Morel, 2012). In addition to the physical task context, it is also crucial to view individuals’ actions over video (Daly-Jones, Monk, & Watts, 1998). The more difficult it is to explain a task verbally, the more beneficial the shared view becomes. Clark and Krych (2004), Gergle, Kraut, and Fussell (2004), Fussell, Kraut, and Siegel (2000) and Kraut, Gergle, and Fussell (2002) studied the performance of collaborators who had a shared view and concluded that it was consistently and considerably faster. Australasian Journal of Educational Technology, 2017, 33(6). 13 Augmented video stream with pointing and drawing It is well known that technology-enhanced social interaction may support knowledge exchange and sharing among peers (Pink et al., 2016; Welsh, Wanberg, Brown, & Simmering, 2003). With respect to communication related to actual tasks, AR’s capability to enhance the communication that typically occurs via face-to-face interactions can reduce the separation between the situation and the task itself, while supporting live interaction with and around relevant artefacts (Lukosch, Billinghurst, Alem, & Kiyokawa, 2015). A typical AR system can improve accuracy and accelerate the work processes of remote collaboration actions. However, the shortcomings of AR tool may be recognised in work situations that require mobility requiring specificity in visual guidance as part of the remote collaboration. The missing sensation of being there can be simulated by having some sort of control over the stream, referred to as telepresence (Gauglitz, Nuernberger, Turk, & Hollerer, 2014b Rae, Mutlu, & Takayama, 2014). Telepresence may enable control of the view within the limits of a selected area guided by the remote person, who can also propose a different viewpoint by directing the camera in the desired location (Jo & Hwang, 2013). Recognising this need, Bauer, Kortuem, and Segall (1999) tested an AR teleporter for remote communication purposes. One could say that their tool was a predecessor to modern AR tools, with features for digital pointing in addition to verbal instruction. Their hypothesis concerning the use of pointing with the tool in the case of expert consultation communication was confirmed with 99% of case use. They also found that situations that used pointing had considerably lower verbal communication. They concluded that “[p]ointing will be the most decisive communication element. In other words, the expert will use pointing gestures first and foremost and support them by verbal explanations, not the other way around” (Bauer et al., 1999, p.152). Considering the issues and opportunities mentioned above, it is possible to claim that remote collaboration can become more immersive and functional when AR is exploited to enhance the experiences of sharing, exploring, referencing and manipulating the physical task context. SoAR — Social augmented reality app SoAR is a mobile app that enhances video calls for the purposes of asking questions and providing guidance in context-dependent work situations. The design concept of SoAR was created as an outcome of a contextual inquiry in the SME (small and medium sized enterprises) sector in the construction industry and then developed within the Learning Layers project. The design and development process followed the research-based design method (Leinonen, 2010; Leinonen, Toikkanen, & Silvast, 2008), such that the SoAR app underwent several iterative cycles via a participatory design process. The final app prototype—the version used in this study—seeks to address and resolve the communication challenges of workers in the construction sector within the limitations of existing AR technology. A deeper look into the history of the prototype’s design and development process is presented in an earlier publication (Pejoska et al., 2016). As a solution, SoAR offers features for live visual assessment, remote augmented communication and enhanced collaboration with pointing and drawing. Because of its ability to alter augmented reality, the app’s core function could be also referred to as mixed reality, as defined by Milgram and Kishino (1994). In the initial design brief, we focused on communication challenges common to the functioning of new workers at a work site. We also considered the generation gap, as we noticed a difference between older workers and younger ones. Specifically, the older workers had considerable knowledge to pass on, while the younger workers had different methods of communicating via smart phones. In sum, the process of designing SoAR involved several different considerations. In essence, SoAR is an open source, Android-based mobile app that is entirely free to use. Once an individual registers as a user, the app enables access to video calling for any contacts from the user’s phone contact list. The video stream shares what each participant is seeing (i.e., the back camera view) rather than a view of the participant’s face (i.e., the front camera view). The audio can be muted from either side if necessary, and either of the participants in a call can choose to view the current view or to switch the shared view to the back camera view at any point. The stream can be also paused, enabling a static view frame. The participants can draw on top of the live stream or the static frame with their finger or use predetermined shapes (e.g., pointers, circles etc.) as a means of communication. Australasian Journal of Educational Technology, 2017, 33(6). 14 Prototype design and development SoAR was created following a contextual inquiry in the construction sector in Finland. The proof-of- concept prototype sought to make a functional vision-sharing feature, which was developed in a web browser using the WebRTC protocol (Figure 1). The result was essentially a video calling system that enabled augmented interactions on top of the live video stream and allowed participants to communicate by drawing on the video call (in addition to communicating verbally and visually) (Pejoska et al., 2016). Figure 1. The vision sharing feature of SoAR The second SoAR prototype was built using Apache Cordova in order to make effective use of Android’s native components. Before using SoAR in an actual work context, we conducted usability tests in lab conditions to make sure that the prototype’s user interface would work as intended: to actually advance collaboration instead of being a disturbance. Following these tests, we engaged in a brief product design phase, during which the usability test findings were analysed, bugs and design flaws were fixed and the in- call user interface was redesigned. The resulting call view in the field study phase is pictured in Figure 2. Figure 2. SoAR call view in the field study phase Australasian Journal of Educational Technology, 2017, 33(6). 15 Research design We aimed to explore how an AR communication tool, SoAR, can change communication and collaboration practices for informal learning at work in the contexts of construction and facility maintenance. To achieve this purpose, we conducted three sets of field studies in which SoAR was used in real working situations and environments (Table 1). At the field study sites, we investigated: a) existing communication practices at the selected working sites, b) employees’ expectations for an AR communication tool, and c) the advantages and challenges of AR at work. To evaluate the potential of SoAR to improve efficiency at the studied working sites, we investigated existing communication practices and workers’ expectations for AR in order to understand the sites’ socio- cultural realities, work cultures and environments. For this purpose, we collected data from semi-structured and themed interviews, carried out job shadowing sessions, and implemented test periods with employees as the application end users. Pre-interviews were conducted to map the types of work contexts in which SoAR would be introduced and their related communication needs. Final interviews were conducted after the test period to collect feedback on the usefulness of the app in the line of work. To collect practical information about the day-to-day communication practices and advantages and challenges of AR at work, we conducted job-shadowing sessions. Shadowing can be described as an observational field study of an informant’s interactions and communication in a mobile work context with the goal of understanding the informant’s actions as a whole (Hyysalo, 2009). The informant allows a researcher to monitor the course of his workday and gives descriptions of his actions as the day progresses. We followed the same process with each field test case. The pre-interviews and job shadowing sessions took place at the beginning of each test period. After that, the participants used SoAR independently. The test period was followed by final interviews. Table 1 Field study participants and research data Field study site Number of participants Age, gender Timeframe Data collecting methods Research data Facility maintenance services 2 35 – 45, 1 female, 1 male 8 Feb – 9 Jun 2016 pre-interviews transcribed audio recordings - 50 minutes for 2 workers job shadowing field notes, photo documentation – 4 hours per worker post-interviews transcribed audio recordings - 32 minutes for 2 workers Construction site 2 25 – 40, 2 male 1 Mar - 3 May 2016 pre-pair- interviews transcribed audio recordings - 56 minutes for 3 workers job shadowing field notes, photo documentation - 4h/worker Australasian Journal of Educational Technology, 2017, 33(6). 16 field test recording phone calls with SoAR - for 18 work days post-pair- interview transcribed audio recordings - 28 minutes (surveyor, foreman) Construction training centre 17 16 – 40, 17 male 1 Mar – 3 May 2016 group-interview transcribed audio recordings - 32 minutes field test Participatory observation and facilitating SoAR calls, photo and video documentation – 4 hours for the group The research data (i.e., recordings and field notes) from the interviews and the job shadowing sessions were transcribed and analysed thematically, focusing on the themes of current communication practices, expectations of AR and possible advantages and challenges of AR at work. The analysis followed a qualitative approach and sought to achieve a comprehensive understanding of the data. We first analysed each of three field study sites separately by identifying incidents under the various themes and uncovering the special characteristics of each site in terms of communication and collaboration practices. We then compared the three sites in order to draw conclusions concerning how AR tools could benefit communication and remote collaboration practices at work. Field study sites We conducted field studies in three different working environments selected to represent the variety of workplaces in the focal industries. Managers and workers at all three field study sites—a facility maintenance services site, a construction training centre and a construction site—were interested in experimenting with new communication practices involving digital technology to improve collaboration and informal learning practices at work. Field study 1: Facility Maintenance Services—ISS Facility Maintenance, Espoo, Finland The facility maintenance services site was a field study site that extended the use of SoAR from a physical working environment to services related to infrastructure and facilities. ISS Palvelut is the Finnish branch of the international ISS group, which offers a range of facility services. The field study was conducted over a four-month period of time with a services lobby attendant and an ISS maintenance person who was responsible for location facilities. The lobby attendants primarily served the users of the building by monitoring the accessibility, safety, and condition of the facility and its movables. They provided visitors guidance and assistance with presentation and printing technologies and space reservations, while also promoting the energy efficient use of the facility. Facility maintenance personnel were responsible for handling service calls and routine tasks related to selecting facilities located close to one another. The routine tasks included, for example, reparations, taking care of outdoor areas, taking monthly electricity and water readings, and doing rounds of the facilities. The facility maintenance personnel received their service calls through several facility management systems and operated on the basis of information received from different building automation systems. Maintenance personnel also often worked on call for additional service areas. In the beginning of the field tests, we conducted 1 hour long introductory interviews with individual users, where we mapped the existing communication practices and their line of work. Both of the users were Australasian Journal of Educational Technology, 2017, 33(6). 17 involved in job-shadowing sessions for 4 hours and the field use of the SoAR app lasted for 2 weeks. In the end, we interviewed the maintenance personnel about their user experiences in the field test period. Field study 2: Construction training centre—BauABC Rostrup, Germany One of the two field study sites in the construction trade industry was BauABC Rostrup, which is one of the largest construction training centres in Germany. At BauABC Rostrup, apprentices from various building occupations take part in initial training before learning on-the-job in construction companies and then later returning for further training periods before graduating to the profession. The operator apprentices at BauABC comprised a group of 17 individuals with different backgrounds. Due to the limited schedule of the Learning Layers project, the field study was conducted as a fully facilitated 1 day test in a practical education setting instead of an actual work context. The test session comprised a short, small-groups introduction to how SoAR works and the actual field study. The apprentices were working in small groups that were geographically dispersed across the facility grounds, and their tasks included using excavators and installing sheet piling into the ground. The apprentices conducted SoAR calls within a 4-hour period with their instructor to ask for advice and to report on how they were doing with the tasks (Figure 3). This participatory observation and facilitation was documented with photos and short videos. Finally, a group interview was conducted to obtain information about the user experiences in the field test period. Figure 3. Field case study at BauABC: Even though the apprentice is aware that missing screws are probably not a serious problem, he decides to consult the instructor over SoAR. Field study 3: Construction site—Skanska, Tampere, Finland The other field study site in the construction industry was Skanska, which is one of the biggest construction companies in Finland and an international project development group. One construction surveyor and one foreman participated in the study at a construction site where they used SoAR independently. The construction surveyor ensured that all structures at the construction site were built on the exact locations specified in the construction plan. The surveyor’s tasks included staking out reference points at the site and taking correction measurements using a total station in conjunction with modelling software. The surveyor who tested SoAR was working at a site in the foundation phase, and his tasks included measuring and staking out the locations of the building’s footing. The construction foremen on the site were responsible for dividing labour and monitoring the overall progress of the construction work. The foreman who tested SoAR supervised the interior phase of one of the apartment blocks during the test period. Australasian Journal of Educational Technology, 2017, 33(6). 18 The surveyor and foreman took part in a 1 hour long introductory pair interview for mapping communication practices and the line of work. The field study tasks included measuring and staking the locations of the current building using SoAR as an assistive tool. The field study activities took place over 18 working days during a 2-month period of time, while the job shadowing lasted for 4 hours per worker. In the end, a pair interview was conducted about their user experiences in the field test period. Results With the data we conducted a thematic analysis of the selected relevant themes in the context, in a qualitative approach. The AR communication tool SoAR exhibited the potential to improve collaboration in the contexts of construction sites in the construction, construction training and facility maintenance services industries. The findings reveal current communication practices and illustrate participants’ expectations of new practices using SoAR. The field study experiences suggested ways that SoAR could improve remote collaboration in the construction and facility maintenance services trades. Current communication practices Mobile devices, and specifically smartphones, have long been standard tools in most lines of work. Combined with the development of web-based services, the shift to smartphones has increased the number of communication and work management systems that can be used on multiple platforms. In relation to the increased use of smart mobile phones at work, we conducted a background study of the field study participants to understand their current remote communication practices with smart phones. From our inquiries we found the following use of remote communication tools on the field. Phone calling In the construction supervision and facility services industry, the mobile phone is the most commonly used remote communication channel and a common tool that nearly everyone is comfortable using for calling and texting. Work-related calls typically have a strong influence on the course of the work day, as evidenced by our job shadowing of a maintenance person in ISS, who received approximately 20 work-related phone calls during a 4-hour period. Despite the culture of calling, long and complex supply chains often make communication difficult. For example, a construction project involves many subcontractors, some of which may change over the course of the project. Each of these contractors has a different work culture. These differences create communication difficulties, which sometimes lead to failures of task management, and responsibility for communication among parties is often unclear. Similar challenges are faced in the facility maintenance sector, in which properties are commonly owned by several property owners and worked on by several content/service developers and subcontractors. These communication challenges in supply chain management are the most-cited risk factors in the construction industry (Aloini, Dulmin, Mininno, & Pontucelli, 2012). Instant messaging culture In facility and maintenance services, mobile communication tools are central to the work management process. Our field studies showed that, in addition to using phones and several facility management systems for monitoring and notifications, many workers had self-initiatively adopted the instant messaging app WhatsApp in the workplace, drawing from positive experiences in their private lives. In practice, the sort of live interaction that these tools provide filled the gap created by other communication tools. WhatsApp, for example, offers features for sharing visual content, such as photos or videos, in direct individual and group channels. We found that smart mobiles, and particularly tablets, were an asset for surveyors who needed portable, up-to-date digital plans at construction sites. The need for such devices was not shared by other workers. As mentioned above, WhatsApp was a popular choice for communicating and sharing views among the BauABC workers. The foremen, it’ll take them hours if they go on searching for some information. But the surveyor has it all in his head—measurements, images, and structural engineering data—so he’s the one to ask from. Plumbers, electricians, they always ask the surveyor. Several occupational safety regulations have made smart mobile use at construction sites challenging. For example, if an employee from Skanska’s found it necessary to make a call, he was required to step away to a safe distance from passengers and machinery. BauABC’s workers were often advised not to use smart Australasian Journal of Educational Technology, 2017, 33(6). 19 phones by older workers, although the younger ones optimistically and persistently talked to their older colleagues about the necessity of mobile use. This attitude towards technology, in which informal communication using mobile devices is perceived as disturbing the workflow, fails to embrace technology’s ability to generate social support, a sense of connectedness and new common ground (see Zhao & Rosson, 2009). The outcomes of our study suggest that the construction supervision and facility services industries should harness improvements in communication and thus support informal learning in the workplace. Exchanging files and photos Across all of the participating organisations, the most common reason to take photos was to document problems and failures. Photos were also used to support workers’ personal memory, to some degree, to exchange views of a particular situation via information systems, instant messaging or email. Face to face problem solving Given the importance of face-to-face collaboration in physical work settings in terms of problem solving and strengthening interpersonal work relations, it was essential to recognise the communication patterns that required improvement and those that did not. For example, colleague or client meetings are important for reasons other than information-sharing: specifically, they also support the building of trust among parties. Many examples of this sort were found on the Skanska construction site, where inspection rounds by the foreman and regular check-ups with the workers were considered to be high priority. We noticed similar physical relationships in situations involving ISS service calls, finding that clients often preferred to address the security person rather than the maintenance person simply because the security person was physically there. Face-to-face collaboration was easier because workers could easily point to or indicate a problem or solution using gestures. Unlike existing communication solutions, SoAR supports remote gesturing and pointing over a shared view. Though the tool does not replace the relationship-strengthening function of a physical presence, the test subjects agreed that they see it as a potentially powerful tool to use at work for both the practices proposed here and several others. Expectations for SoAR calls Facility maintenance services workers would often make phone calls to communicate and find solutions in specific situations, and they often used photos to document these problems. In a hypothetical help-seeking situation, SoAR could replace phone calls by supporting the sharing of photos with video calls over a shared view. All of the interviewed field study participants identified various situations in which SoAR would assist the process of remote collaboration while reducing handling time. Let’s say there’s some electrical fault, so it goes like this. First, we tell [maintenance]: “Yeah, we’ve checked the fuses. The socket doesn’t work.” Maybe the next day, the maintenance guy comes over and goes: “Yeah, that’s right, doesn’t work.” And he’ll call an electrician, which will take another two to three days until he comes. That process could be faster with the video call ... I could say: “Look at this, here are the fuses and everything looks ok.” He’d see what I see and do. Because [otherwise], he’ll have to come over, like maybe I didn’t know where to look. --- But during a video call, he could say: “Hey, there’s another fuse box there and there, go check that out.” Not that we’d take [the work] away from them, but it would serve the building better. If it’s some quick fuse or something and it takes days for someone to come and figure that out, of course it’s better if I can find the fault and fix it. The proposed uses included the remote identification of problems and solutions using SoAR’s pointing over a shared view feature. Using SoAR, a maintenance person could decide whether his physical presence is needed or whether to forward the problem to, for example, a plumber or electrician. Our examinations focused on the communication between the maintenance person and the lobby attendant, which allowed us to identify possible uses for the tool both in-house and as a supplement to the existing facility management system. From the explored situations, we also identified potential future cases for improvement. For example, collaborations with subcontractors could be especially beneficial for saving time. Australasian Journal of Educational Technology, 2017, 33(6). 20 At BauABC, we started the field studies with the initial goal of using SoAR to extend the view of the construction machine operators (Figures 4 and 5). Apprentices often have to give instructions to the operator from outside of the cabin in order to avoid blind spots. For example, when an operator is installing sheet piling into the soil at a certain angle or manoeuvring the shovel of an excavator, he is usually instructed via walkie-talkies and/or pointing gestures from outside. A mobile device with SoAR installed inside the cabin could allow the apprentice to follow instructional drawings and a live view of the outside. Figure 4. Field case study at BauABC. At the beginning of their exercise, the apprentice group working with sheet piling checks the situation for issues by pointing to and drawing possible problems. Figure 5. Field case study at BauABC: The cause of noise needs to be determined. The instructor and the group are looking at the joints and pointing to them using the SoAR drawing feature. The instructor asks the group of apprentices whether the procedure of lubricating the joints has been followed. We’re currently at the mini excavator and making the 10-meter trench, and we have a small question about it: namely, the joint up there. We oiled it yesterday, but it squeals like a pig. Australasian Journal of Educational Technology, 2017, 33(6). 21 At the construction site, the field study participants suggested that the frame phase of a building project would benefit the most from the use of SoAR. The frame phase is when all contractors need to be ready for collaboration and when the surveyor detects inconsistencies and communicates them with the supervisors. The use of SoAR could bridge the long distances (50 metres) from the office to the site and allow foremen and surveyors to monitor the site together and the workers together. It was also suggested that SoAR be used as a direct communication link between the structural engineering agency and the surveyor. At all of the field study sites, SoAR was expected to reduce the need for immediate presence. Though SoAR could be a platform for solving problems faster, the participants’ expectations highlighted improving communication in the workplace rather than enabling savings for the employer. This is because the study sought to understand the employees’ point of view, and the employees considered saving time to be the most important way SoAR could make their work more manageable. SoAR: Advantages and challenges The results of the field studies imply that SoAR is both usable and useful in various collaborative situations in facility maintenance, construction and construction training work (Figures 6 and 7). At the ISS facility maintenance site, SoAR was planned to be used to report service needs. The field studies suffered technical difficulties, and some of SoAR’s communication channels had to be tested twice due to the obligation to also use the current task management system. Despite this, the staff saw SoAR as useful and felt that it had the potential to replace regular phone calls, support remote training presentations, allow the reporting of failures in acute situations and allow workers to quickly recognise malfunctions, thus preventing serious damage. All field study participants considered it an easy app to use. They noted that SoAR would be particularly useful for handling service calls related to a new building and the guarantee period, when many fault reports are typically made through the facility maintenance company. For example, maintenance personnel and contractors could use SoAR to effectively identify and address possible issues. I believe we would use [SoAR] between the foreman, for example, and us, or us and a subcontractor, because it will reduce the time a job takes by an hour or two. If we can show [the issue] during a video call, it's enough for them to realise what materials they need. Figure 6. Field case study at BauABC: The apprentices show a close up of the joints and cylinders to the instructor remotely via SoAR, and the instructor notices that the cylinder is causing the noise. After inspecting under the cylinder, he instructs the apprentices to lubricate it again. Australasian Journal of Educational Technology, 2017, 33(6). 22 The maintenance personnel commented on the motivational aspects of using the AR tool, but also recognised more realistic prospects related to supply chain management and internal communication. For example, the apprentices at BauABC thought that SoAR could be used as a visual aid in tasks related to operating a machine and in construction training settings involving the remote teaching of independent or group instructional exercises. Figure 7. Field case study at BauABC: Examining the state of the excavator. The instructor writes instructions using graphs of the tasks. After explaining his instructions for the apprentices verbally, he uses SoAR’s drawing features to find out how many of the tasks the apprentices have already completed. The apprentices also believed that SoAR could be used to overcome other problems in their work, such as situations in which they encountered technical faults that needed to be communicated to external personnel such as repair shops. The participants at Skanska were optimistic that SoAR could improve communication and efficiency during the frame phase if the app were widely adopted by the work community. To be able to make a video call straight to the structural engineer or to someone, to be able to present the problems to someone straight from the field ... [SoAR] would make it easier. No need to make phone calls or send emails—those may be replied at some point, if ever. It was clear that an app would not replace face-to-face communication entirely. Rather, SoAR provided workers with an alternative solution for assisting processes and monitoring status. SoAR could also be used to monitor occupational safety, since foremen often spend considerable time training new subcontractors. By using video streaming technology (rather than still images) to observe structures from every angle, workers could build more accurate understandings of their construction sites. One of the challenges facing the future use of AR mobile apps in the construction and maintenance industry is that of harsh weather conditions. Research findings on the challenges of mobile use in the construction work environment have been confirmed in earlier field studies (Bauters, Purma, & Leinonen, 2014; Pejoska et al., 2016), which have identified building noise, screen visibility under direct sunlight, rain, low temperatures, protection-ware, and connectivity problems as potential issues affecting AR use in construction and maintenance. It is important to note that most of these concerns relate to the use of mobile phones in general, not to SoAR-specific complications. Still, the apprentices at BauABC were optimistic about finding solutions for most of these possible issues, for example, they suggested wearing plug headphones under their earmuffs in order to make calls in noisy situations. Furthermore, though large Australasian Journal of Educational Technology, 2017, 33(6). 23 construction sites like Skanska’s typically lack network infrastructure until the buildings are ready for their intended use, such connectivity issues will soon be a thing of the past. In sum, though mobile phones are a viable and robust technology capable of enabling shared-view communication in the construction sector, their use on construction sites faces several limitations. In addition to context-specific challenges, it is also necessary to consider other issues related to the wider adoption of SoAR. In large companies like Skanska, ISS maintenance, and BauABC, implementing new tools implies training staff, which can represent a considerable investment. Furthermore, some companies, such as Skanska, have adopted specific operating systems as their official platforms. Thus, to be useful in any given site, SoAR may need to work on both iOS and Windows. These upgrades represent possible avenues for future SoAR development. Discussion This study has investigated mobile augmented communication for remote collaboration and explored how video calls augmented with SoAR could change communication and collaboration practices for informal learning in the physical work context. Since the participants did not feel the need to adapt SoAR to their tasks, it can be inferred that the app is sufficiently generic to be used in other lines of physical work. The examination of existing communication practices revealed that phone calls were the most important method of communicating in acute situations in all participating organisations. Since the mental model of SoAR resembles that of a phone call and since the purpose of SoAR is to enhance remote collaboration in immediate situations, it can be argued that SoAR is an effective substitute for audio calls. However, although SoAR could theoretically replace audio calls, it cannot yet replace face-to-face communication. SoAR supports situation awareness when it is not possible to meet relevant co-workers or contacts in person. This may happen, for example, when a worker must remain at her post, when it is dangerous to leave a situation unattended or when physical distance or a task’s timeframe makes co-located collaboration impossible. SoAR and other AR apps may support a necessary adjustment phase in the shift toward completely different communication interfaces and mental models, such as wearable AR hardware. Drawing on a video stream is both effective and preferable to communicating over video and audio. Pointing by drawing reduces misunderstandings and builds common ground quicker than relying solely on a shared view. For this reason, it was interesting that none of the field study participants had ever used or experienced others using video conversation apps in the work environment. In the future, it may be interesting to explore the reasons video call applications are still largely seen as conference tools and not as aids to physical work. On the other hand, independent SoAR use also yielded information on why using new applications and tools is so challenging. Though SoAR was considered an easy app to use, numerous everyday obstacles disrupted the testing of the app, particularly since there was no routine or compelling need to make SoAR calls. It is necessary to be aware of such hurdles during the exploitation phase of SoAR or any similar work- related application. The next SoAR research phase should extend the app’s use in work settings. For example, SoAR could be further tested by engaging a group of workers who have already established communication patterns among themselves. During the test phase, the participants would make SoAR calls instead of audio calls in as many situations as possible without disturbing the normal workflow. Such extended field studies would produce larger-scale knowledge on how SoAR works in acute work situations in new working contexts. This study has shown that visual augmented communication benefits collaboration at work and that its regular use could lead to more frequent collaboration in acute situations and planned processes. SoAR and similar technologies may enable alternative training opportunities for remote participants, such as machine operator students and others. Such functions may be especially useful in information-intensive tasks involving significant human decisions (Wang & Dunston, 2007). Although the participants faced several challenges while using SoAR, these had more to do with the facilitating conditions and the technical state of the prototype than with the app itself or its ideal use. Effective remote collaboration could enhance the culture of mutual assistance, improve the sharing of knowledge acquired by individual workers and mitigate Australasian Journal of Educational Technology, 2017, 33(6). 24 differences in employees’ professional backgrounds and vocabularies. All of these outcomes can be expected to improve practices of informal learning at work. Conclusion This study has explored whether augmented video calls are useful in facilitating communication and remote collaboration in physical work contexts in the construction and facility maintenance services industries. The field study outcomes suggest that the SoAR app is a potential solution, especially for acute and ad hoc work situations. In sum, SoAR could improve communication in quality and supply chain management work processes in the construction and facility maintenance sectors. Acknowledgement We acknowledge EU/FP7’s financial support of the Learning Layers project and thank the technical developers Matti Jokitulppo and Leo Nikkilä for their contributions to the SoAR prototypes. References Aloini, D., Dulmin, R., Mininno, V., & Ponticelli, S. (2012). Supply chain management: A review of implementation risks in the construction industry. Business Process Management Journal, 18(5), 735– 761. https://doi.org/10.1108/14637151211270135 Baird, K. M., & Barfield, W. (1999). Evaluating the effectiveness of augmented reality displays for a manual assembly task. Virtual Reality, 4(4), 250–259. https://doi.org/10.1007/BF01421808 Baker, M., Hansen, T., Joiner, R., & Traum, D. (1999). The role of grounding in collaborative learning tasks. In P. Dillenbourg (Ed.), Collaborative Learning: Cognitive and computational approaches (pp. 31-63; 223-225). Elsevier Science. Bauer, M., Kortuem, G., & Segall, Z. (1999). ‘Where are you pointing at?’ A study of remote collaboration in a wearable videoconference system. Proceedings of The Third International Symposium on Wearable Computers, Digest of Papers, Washington, 151–158. Bauters, M., Purma, J., & Leinonen, T. (2014). In-time on-place learning. Proceedings of the 10th International Conference on Mobile Learning, Madrid. IADIS Press. Clark, H. H., & Krych, M. A. (2004). Speaking while monitoring addressees for understanding. Journal of Memory and Language, 50(1), 62–81. https://doi.org/10.1016/j.jml.2003.08.004 Clark, H. H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22(1), 1–39. https://doi.org/10.1016/0010-0277(86)90010-7 Daly-Jones, O., Monk, A., & Watts, L. (1998). Some advantages of video conferencing over high-quality audio conferencing: Fluency and awareness of attentional focus. International Journal of Human- Computer Studies, 49(1), 21–58. https://doi.org/10.1006/ijhc.1998.0195 Dourish, P. (2001). Where the action is. Cambridge: MIT press. https://doi.org/10.1162/leon.2003.36.5.412 Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 32–64. Eraut, M. (2007). Learning from other people in the workplace. Oxford Review of Education, 33(4), 403– 422. https://doi.org/10.1080/03054980701425706 Fussell, S. R., Kraut, R. E., & Siegel, J. (2000). Coordination of communication: Effects of shared visual context on collaborative work. Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, Philadelphia, PA, 21–30. https://doi.org/10.1145/358916.358947 Gauglitz, S., Nuernberger, B., Turk, M., & Höllerer, T. (2014b). In touch with the remote world: Remote collaboration with augmented reality drawings and virtual navigation. Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology, Edinburgh, 197–205. https://doi.org/10.1145/2671015.2671016 Gergle, D., Kraut, R. E., & Fussell, S. R. (2004). Action as language in a shared visual space. Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, Chicago, 487– 496. https://doi.org/10.1145/1031607.1031687 https://doi.org/10.1108/14637151211270135 https://doi.org/10.1007/BF01421808 http://vbn.aau.dk/en/publications/the-role-of-grounding-in-collaborative-learning-tasks(ce803920-9c2e-11db-8ed6-000ea68e967b).html http://vbn.aau.dk/en/publications/the-role-of-grounding-in-collaborative-learning-tasks(ce803920-9c2e-11db-8ed6-000ea68e967b).html https://doi.org/10.1016/j.jml.2003.08.004 https://doi.org/10.1016/0010-0277(86)90010-7 https://doi.org/10.1006/ijhc.1998.0195 https://doi.org/10.1162/leon.2003.36.5.412 https://doi.org/10.1080/03054980701425706 https://doi.org/10.1145/358916.358947 https://doi.org/10.1145/2671015.2671016 https://doi.org/10.1145/1031607.1031687 Australasian Journal of Educational Technology, 2017, 33(6). 25 Gergle, D., Kraut, R. E., & Fussell, S. R. (2013). Using visual information for grounding and awareness in collaborative tasks. Human–Computer Interaction, 28(1), 1–39. Retreieved from http://collablab.northwestern.edu/pubs/HCI2013-GergleKrautFussell- UsingVisualInformationforGroundingandAwarenessinCollaborativeTasks.pdf Hwang, G. J., Shi, Y. R. & Chu, H. C. (2011). A concept map approach to developing collaborative Mindtools for context-aware ubiquitous learning. British Journal of Educational Technology, 42(5), 778-789. https://doi.org/10.1111/j.1467-8535.2010.01102.x Hyysalo, S. (2009). Käyttäjä tuotekehityksessä. Tieto, tutkimus, menetelmät. [The user product development. Knowledge, research, methodology]. Taideteollisen korkeakoulun julkaisu B. 2. Uudistettu laitos. Helsinki: Taideteollinen korkeakoulu. Jo, H., & Hwang, S. (2013, April). Chili: Viewpoint control and on-video drawing for mobile video calls. Proceedings of CHI'13 Extended Abstracts on Human Factors in Computing System, Paris, 1425– 1430. https://doi.org/10.1145/2468356.2468610 Kraut, R. E., Gergle, D., & Fussell, S. R. (2002). The use of visual information in shared visual spaces: Informing the development of virtual co-presence. Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work, New Orleans, LA, 31–40. https//doi.org/10.1145/587078.587084 Leinonen, T. (2010). Designing learning tools. Methodological insights. Helsinki: Aalto University. Leinonen, T., Toikkanen, T., & Silfvast, K. (2008). Software as hypothesis: Research-based design methodology. Proceedings of the Tenth Anniversary Conference on Participatory Design, Bloomington: ACM 61–70.Leskinen, S. (2008). Mobile technology in the Finnish construction industry—Present problems and future challenges. 21st Bled Conference Collaboration: Overcoming Boundaries through Multi-Channel Interaction, Slovinia, 44. Licoppe, C., & Morel, J. (2012). Video-in-interaction: ‘Talking heads’ and the multimodal organization of mobile and Skype video calls. Research on Language & Social Interaction, 45(4), 399–429. https://doi.org/10.1080/08351813.2012.724996 Lukosch, S., Billinghurst, M., Alem, L., & Kiyokawa, K. (2015). Collaboration in augmented reality. Computer Supported Cooperative Work, 24(6), 515–525. https://doi.org/10.1007/s10606-015-9239-0 Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. Institute of Electronics Information and Communication Engineers Transactions on Information and Systems, 77(12), 1321– 1329. Pejoska, J., Bauters, M., Purma, J., & Leinonen, T. (2016). Social augmented reality: Enhancing context‐ dependent communication and informal learning at work. British Journal of Educational Technology, 47(3), 474-483. https://doi.org/10.1111/bjet.12442 Peng, H., Su, Y., Chou, C. & Tsai, C. (2009). Ubiquitous knowledge construction: Mobile learning re‐ defined and a conceptual framework. Innovations in Education and Teaching International, 46(2), 171–183. https://doi.org/10.1080/14703290902843828 Pink, S., Lingard, H., & Harley, J. (2016). Digital pedagogy for safety: The construction site as a collaborative learning environment. Video Journal of Education and Pedagogy, 1(1), 5. https://doi.org/10.1186/s40990-016-0007-y Rae, I., Mutlu, B., & Takayama, L. (2014). Bodies in motion: Mobility, presence, and task awareness in telepresence. Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, Toronto, 2153–2162. https://doi.org/ 10.1145/2556288.2557047 Wang, X., & Dunston, P. S. (2007). Design, strategies, and issues towards an augmented reality-based construction training platform. ITcon, 12, 363–380. Retrieved from http://www.itcon.org/2007/25 Welsh, E. T., Wanberg, C. R., Brown, K. G., & Simmering, M. J. (2003), E-learning: Emerging uses, empirical results and future directions. International Journal of Training and Development, 7, 245– 258. https://doi.org/:10.1046/j.1360-3736.2003.00184.x Zhao, D., & Rosson, M.B. (2009) How and why people Twitter: The role that micro-blogging plays in informal communication at work. Proceedings of the ACM 2009 International Conference on Supporting Group Work, Sanibel Island, 243-252. https://doi.org/10.1145/1531674.1531710 Zollmann, S., Hoppe, C., Kluckner, S., Poglitsch, C., Bischof, H., & Reitmayr, G. (2014) Augmented reality for construction site monitoring and documentation. Proceedings of the IEEE, Piscataway, 137-154. https://doi.org/10.1109/JPROC.2013.2294314 http://collablab.northwestern.edu/pubs/HCI2013-GergleKrautFussell-UsingVisualInformationforGroundingandAwarenessinCollaborativeTasks.pdf http://collablab.northwestern.edu/pubs/HCI2013-GergleKrautFussell-UsingVisualInformationforGroundingandAwarenessinCollaborativeTasks.pdf https://doi.org/10.1111/j.1467-8535.2010.01102.x https://aaltodoc.aalto.fi/handle/123456789/11826 https://aaltodoc.aalto.fi/handle/123456789/11826 http://doi.org/10.1145/2468356.2468610 https://doi.org/10.1145/587078.587084 https://doi.org/10.1080/08351813.2012.724996 https://doi.org/10.1007/s10606-015-9239-0 https://doi.org/10.1111/bjet.12442 https://doi.org/10.1080/14703290902843828 https://doi.org/10.1186/s40990-016-0007-y https://doi.org/%2010.1145/2556288.2557047 http://www.itcon.org/2007/25 https://doi.org/:10.1046/j.1360-3736.2003.00184.x https://doi.org/10.1145/1531674.1531710 http://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=%22Authors%22:.QT.Stefanie%20Zollmann.QT.&newsearch=true http://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=%22Authors%22:.QT.Christof%20Hoppe.QT.&newsearch=true http://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=%22Authors%22:.QT.Stefan%20Kluckner.QT.&newsearch=true http://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=%22Authors%22:.QT.Christian%20Poglitsch.QT.&newsearch=true http://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=%22Authors%22:.QT.Horst%20Bischof.QT.&newsearch=true http://ieeexplore.ieee.org/search/searchresult.jsp?searchWithin=%22Authors%22:.QT.Gerhard%20Reitmayr.QT.&newsearch=true http://ieeexplore.ieee.org/document/6705607/ http://ieeexplore.ieee.org/document/6705607/ http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=5 https://doi.org/10.1109/JPROC.2013.2294314 Australasian Journal of Educational Technology, 2017, 33(6). 26 Corresponding author: Jana Pejoska-Laajola, jana.pejoska@aalto.fi Australasian Journal of Educational Technology © 2017. Please cite as: Pejoska-Laajola, J., Reponen, S., Virnes, M., & Leinonen, T. (2017). Mobile augmented communication for remote collaboration in a physical work context. Australasian Journal of Educational Technology, 33(6), 11-26. https://doi.org/10.14742/ajet.3622 mailto:jana.pejoska@aalto.fi https://doi.org/10.14742/ajet.3622 Introduction Augmented communication for remote collaboration Collaboration over a shared view Augmented video stream with pointing and drawing SoAR — Social augmented reality app Research design Field study sites Results Current communication practices Phone calling Instant messaging culture Exchanging files and photos Face to face problem solving Expectations for SoAR calls Discussion Conclusion Acknowledgement References