Australasian Journal of Educational Technology, 2022, 38(6). 150 Smart Groups: A system to orchestrate collaboration in hybrid learning environments. A simulation study Adrián Carruana Martín, Carlos Alario-Hoyos, Carlos Delgado Kloos Universidad Carlos III de Madrid, Spain COVID-19 has brought new hybrid learning environments with some students in the classroom and some others online, synchronously, due to the needs of social distancing. These new hybrid learning environments pose new challenges, for example for group collaboration. This paper presents Smart Groups, a system aimed at helping teachers to orchestrate collaboration in hybrid learning environments and assesses its usability and usefulness through a simulation study. Smart Groups identifies the students that are in the classroom and online, automates the creation of groups (recommending collaborative learning flow patterns to teachers and considering the previous work done by students), supports the communication among students and the use of additional tools and resources for collaboration, and helps maintain the safety distance among the students who are in the classroom. The usability of Smart Groups has been assessed through a mock-up by 60 users (41 students and 19 teachers) with the system usability scale (SUS) obtaining good results (mean = 75.47, standard deviation = 14.95, median = 76.25). A subgroup (10 teachers out of 19) carried out follow-up interviews using the technology acceptance model (TAM) and highlighted the usefulness of Smart Groups to orchestrate collaboration in hybrid learning environments. Implications for practice or policy: ● Smart Groups supports teachers in the orchestration of groups in hybrid learning environments. ● Smart Groups facilitates group coordination and communication among students. ● Smart Groups helps maintain the safety distance. Keywords: hybrid learning, collaborative learning, orchestration, smart learning environment, beacons, positioning Introduction COVID-19 has caused important changes in education, particularly in the way of teaching and learning, forcing the transformation of traditional face-to-face learning environments into online and/or hybrid learning environments, where, for example, some students attend the lecture in the classroom and others follow it online at the same time (Raes et al., 2020). Nevertheless, it is important to be careful when implementing these changes so that they do not negatively affect learning (Williamson et al., 2020). Some of the main challenges derived from the abovementioned fast transformation include: the coordination between teacher and students; the communication among students and with the teacher; the collaboration of students in groups; and the management of new tools (Al Lily et al., 2020; Chang & Fang, 2020). These problems may be aggravated by the changing educational environments, as may be the case with the new models for hybrid learning environments that have emerged. These changing educational environments may benefit from advanced technologies, becoming smart learning environments (SLEs) (Hwang, 2014). Although there are certain interpretations of the term SLE (Carruana Martín et al., 2019), a commonly accepted definition refers to “the technology-supported learning environments that make adaptations and provide appropriate support (e.g., guidance, feedback, hints, or tools) in the right places and at the right time to learners based on their individual needs, which might be determined via analysing learners’ behaviours, performance and the online and real-world contexts in which learners are situated” (Hwang, 2014, p. 5). Applications of SLEs include, for example, the promotion of collaboration between students (Tabuenca et al., 2020). Collaboration and collaborative learning have been extensively used in traditional educational environments (Gokhale, 1995). Nevertheless, the orchestration of collaborative tasks is often a time-consuming and sometimes complex task for the teacher (Dillenbourg & Jermann, 2010), and where advanced technologies can be helpful. One important decision when orchestrating collaborative tasks refers to group creation and management. There are different ways to Australasian Journal of Educational Technology, 2022, 38(6). 151 create groups, for example, assigning group members randomly or based on their previous knowledge, ability or performance in the course (van der Laan Smith & Spindle, 2007), that is, homogeneous groups with the same level of knowledge, ability, or previous performance, or heterogeneous groups with different levels of knowledge, ability, or previous performances (Rubio-Fernández et al., 2019). Although there are an important number of articles that addressed the topic of orchestration of collaboration (Dillenbourg & Jermann, 2010; Prieto et al., 2011) there is an open line of research as a result of COVID-19, since collaboration now can take place in hybrid learning environments (with some students in the classroom and others connected online) and where safety distance must be respected in the case of students (and the teacher) who are in the classroom. Therefore, this article addresses the combination of these three issues: orchestration of collaborative activities in hybrid learning environments that require safety distance as a consequence of COVID-19. The COVID-19 situation being the main novelty of this work compared to the existing related literature. The different problems and gaps of the abovementioned research (Dillenbourg & Jermann, 2010; Prieto et al. 2011) have been taken into account in the creation of Smart Groups, a system developed to address the combination of these three issues, and for which a simulation study is conducted with the aim to assess its usability and usefulness. Smart Groups facilitates the orchestration of collaborative learning tasks, particularly in hybrid learning environments, with some students in the classroom and some others online, synchronously, turning them into Smart Hybrid Learning Environments. More specifically, Smart Groups makes it easier for the teacher to create groups taking into account who is in the classroom and who is online, by automatically identifying onsite students through beacons located inside the classroom (Bae & Cho, 2015). In addition, the teacher receives recommendations of collaborative learning flow patterns (CLFPs) at the time of creating or modifying the groups (Hernández Leo, Asensio-Pérez, & Dimitriadis, 2005). Moreover, Smart Groups also facilitates resource sharing and communication among students in groups and with the teacher. Finally, Smart Groups uses the sensors on the students and teacher's smartphones to position them and warn them if the required safety distance is not respected. A clear use of Smart Groups is in the case of institutions which are following a hybrid approach for synchronous lectures in the 2021/2022 school year, with some students attending onsite for one session/week and online for the following session/week, and vice versa, in order to meet the new requirements of capacity with social distancing in the classroom. In this scenario, Smart Groups supports the teacher in the creation and management of groups without the need to know who is in the classroom and who is online. In addition, Smart Groups also supports the teacher with tools for the communication with the students, and the management of groups, even when students change groups during an activity, and all this while supervising that everybody respects the safety distance. The following section analyses the literature related to collaboration in hybrid learning environments and positioning of students and teachers in the classroom. Next, the design of Smart Groups and the components that make up the system, as well as its architecture, are presented. The methodology is then described. The results obtained from the evaluation of Smart Groups are presented and analysed. The discussion on the results and the answers to the research questions are then presented. The article finishes by drawing conclusions and pointing out some future lines of research. Research problem and objectives This article presents Smart Groups from a technical and functional perspective and reports the results of a simulation study carried out with 60 users who have used a Smart Groups mock-up. The objective of this simulation study was to assess the usability and usefulness of Smart Groups in a context of supporting the orchestration of collaboration in hybrid learning environments. Therefore, the following two questions were posed from a research perspective: RQ1: Is Smart Groups a usable system to support collaboration in hybrid learning environments? RQ2: Is Smart Group useful to orchestrate collaboration in hybrid learning environments? Related work The review of the related literature for the creation of Smart Groups began with an analysis of the main theoretical frameworks in the field of collaborative learning. These frameworks were those defined by Prieto et al. (2011) and Prieto et al. (2015) on orchestration of collaboration, and Hwang (2014) on smart Australasian Journal of Educational Technology, 2022, 38(6). 152 learning environments. Challenges in education derived from the COVID-19 pandemic were then incorporated, including the new models for hybrid education. The theoretical framework by Pérez- Sanagustín et al. (2017) was used to develop the concept of hybrid education. These articles served as the basis to extract related references and search for current literature on these topics. Theoretical framework The orchestration of collaboration is a topic that has been covered extensively in the literature of collaborative learning (Dillenbourg et al., 2013; Roschelle et al., 2013), and for which several theoretical frameworks have been defined (Prieto et al., 2011; Prieto et al., 2015). The main challenges related to orchestration of collaboration include space management (Nieswandt et al., 2020), class control, and time management (Olsen et al., 2020). These challenges depend on the specific educational context. Many educational institutions are dealing with the particular context of hybrid learning with synchronous lectures followed onsite by some students with others online. It is precisely in this context that positioning in the classroom becomes even more important when collaborating in groups (Martinez-Maldonado, 2019) due to the required safety distance required for COVID-19 for those students who are inside the classroom (Sun & Zhai, 2020). Collaboration and hybrid learning environments The creation and management of groups in collaborative tasks involves several problems, such as the selection of the group structure, or the assignment of group members (Hernández-Leo et al., 2006). For example, Herrera-Pavo (2021) conducted an in-depth study of collaborative learning in virtual environments and found that students' perception of the group influenced collaborative work. Students participating in their study considered it important to share interests with their group mates and to feel that they could be autonomous from the teacher. Another example is the work of Childress and Braswell (2006), who analysed group work in the context of online video games, covering several collaboration techniques. They concluded, supported by previous work, that the way groups were created could have an impact on learning. Azukas (2019) analysed collaborative learning from the teacher’s perspective, concluding that the teacher must know the characteristics of the students when creating the groups to make collaborative learning more effective. Another well-known problem refers to the orchestration of collaborative learning activities and, particularly, the time and effort of the teacher to create and manage the groups of students. For example, Hämäläinen and Oksanen (2012) discussed the tasks and workload needed to orchestrate collaboration from the teacher's point of view and concluded that giving the teacher appropriate tools for orchestration and more information about the related tasks was crucial for an efficient orchestration. CLFPs can be helpful in the orchestration of collaborative learning tasks, including popular patterns such as Jigsaw (Adams, 2013), Pyramid (Manathunga, & Hernández-Leo, 2018), or Think-Pair-Share (Azlina, 2010). Several articles have designed tools to implement these patterns from the design phase of a collaborative learning activity. One of these works is ILDE (Hernández-Leo et al. 2014), which integrates different free and open-source tools for learning design, including tools that facilitate orchestration. Actually, ILDE helps teachers in the creation of groups and allows them to select CLFPs. Another related work is Web College (Villasclaras-Fernández et al., 2013), a tool for supporting teachers in collaborative learning environments, providing recommendations of CLFPs according to the type of task and/or objectives to be achieved with that task. In addition, it is worth noting the work by Rubio-Fernández et al. (2019), which facilitates the creation of homogeneous or heterogeneous groups of students during the onsite phase of the flipped classroom based on students’ work at home in the online phase, watching videos, or doing formative exercises, when available. Collaborative learning has been widely embraced, both in face-to-face and online learning environments. However, little work has been done in hybrid learning environments, such as those where some students attend the lecture in the classroom and others follow it online. In fact, the definition of hybrid learning varies among authors, with some of them using this term as a synonym for blended learning (e.g., Olapiriyakul & Scher, 2006), and others using hybrid learning to refer to the integration massive open online courses (MOOCs) or MOOC-related technologies into a traditional curriculum (Hartono et al., 2018; Pérez-Sanagustín et al., 2017). A number of theoretical frameworks were proposed in relation to hybrid Australasian Journal of Educational Technology, 2022, 38(6). 153 learning (Jamison et al., 2014; Pérez-Sanagustín et al., 2017), and there were publications who addressed collaborative learning and hybrid education, in the broad sense. One of these publications, by Roseth et al. (2013), analysed and explained different tools to be used in some hybrid learning environments to support cooperative, but not collaborative, learning (Dillenbourg, 1999). Another related work by Hrastinski (2008), analysed the advantages and disadvantages of asynchronous and synchronous e-learning. One relevant conclusion is that it is easier to collaborate in environments that alternate between face-to-face and online education (asynchronous environments) than in environments where both face-to-face and online education take place at the same time (synchronous environments). One of the Hrastinski’s conclusions was that there is much to be done in synchronous hybrid learning environments to improve collaborative learning. Another related publication by Bower et al. (2015), analysed the different factors for designing hybrid learning environments. In their conclusions, the authors specified the need for collaboration technologies to be invisible to teachers and students for their use in hybrid learning environments, minimising the exchange of information and interactions required in order not to overload the user. All these publications highlight the need for new solutions to orchestrate and support collaboration in hybrid learning environments, particularly in synchronous ones, such as those which have some students in the classroom and some others online. Furthermore, existing publications analyse different characteristics of hybrid learning environments and provide recommendations, but no single work proposes a solution for the orchestration of collaborative learning in synchronous hybrid learning environments. Positioning in the classroom Positioning in the classroom can be used to obtain information about the teacher, the students, and the environment. This information can be used for several purposes, such as attendance control. Zorić et al. (2019) developed a mobile application that together with beacons detected the presence of students in the class and confirmed their attendance. They concluded that their application was useful for attendance control and that a manual process would take more time. Bdiwi et al. (2019) developed an architecture that used the signal from a router and radio frequency identification (RFID) sensors to detect student attendance, as well as student performance according to the teacher's position. This analysis concluded that students were more involved the closer the teacher was. Other related publications addressed access control to the classroom. For example, Mittal et al. (2015) developed a fingerprint-based system to guarantee access and manage student attendance. They specified the importance of the security provided by their access control system. Subedi et al. (2016) developed another access control system with beacons covering several zones. Another related challenge is assigning resources depending on students’ position in the classroom. For example, Guo et al. (2015) developed a system that detected the position of the student through the signal of routers. Once they detected the students in the classroom, they gave them access to the material that the teacher had previously specified. Guo et al. (2015) highlighted the high accuracy of their system in locating students. Another example related to providing materials or resources based on the position of the student by Bayğın et al. (2021), developed a student access authorisation system for smart library environments. This system offers information to students depending on their position, which is detected through RFID cards that the students carry with them. With their system, Bayğın et al. (2021) streamlined, filtered, and facilitated student access to information. Finally, Hauge et al. (2017) developed a game-based learning system that used beacons to create access zones. When students accessed a designated area, they received access to a new mini-game. Hauge et al. (2017) particularly highlighted the portability of the system to be taken to other locations. Although the issue of positioning in the classroom is being addressed in the literature, it is possible to propose new applications, especially in an environment where safety distance is needed due to COVID-19. Among the existing solutions which could also be applied to maintain the safety distance, it is worth highlighting the use of beacons for attendance control and detection by zones, while the WiFi signal can provide greater accuracy on teacher and students’ positions. Design Smart Groups is a system designed to support collaboration and communication among students, particularly in synchronous hybrid learning environments (with some students in the classroom and some others online, simultaneously). It uses positioning to deal with the required safety distance among students Australasian Journal of Educational Technology, 2022, 38(6). 154 who are in the classroom (and also with the teacher). This system consists of two mobile applications (APPs) one for the teacher, another for the student, and a set of beacons positioned in the classroom. These APPs are the bridge between the system and the users. Moreover, the system is supported by the sensors of the smartphones where the APPs are installed together with the beacons for positioning tasks. Teacher APP This application aims to enable the teacher to orchestrate collaboration in an efficient and user-friendly way in hybrid learning environments. The APP helps to orchestrate group tasks in a few steps offering the possibility to use CLFPs, under the teacher's supervision. In addition, the APP notifies the teacher in case they do not respect the safety distance with one or more students. The application requires the teacher to log in to their corporate accounts so that their data can be related to their assigned courses and classes (Figure 1a). Once the teacher logs in, their courses and classes will be displayed on the screen (Figure 1b). The teacher can then select the classes in which they want to create a collaborative task in order to define the corresponding groups. (a) (b) Figure 1. First screens of Teacher APP: (a) login screen; (b) the classes available to the teacher When the teacher selects a class, they will be able to review the saved group settings (if any) or to create new group settings (Figure 2a). If the teacher chooses the first option, the screen will display the saved group settings (Figure 2b), with the options that were chosen for that setting (Figure 2c). If the teacher chooses the second option, the screen will show a new menu to create a new group setting. The teacher has several options to choose from. The first is “Name”, where the name of the group configuration is defined. The next one is “Task”, where the teacher marks the characteristics of the task. The third one is “Students per group”, where the teacher specifies the maximum number of students per group. The following is “Dynamic”, where the teacher marks if they would mind making changes to the groups after creating them. The fifth is “Group Type”, where the teacher indicates whether they want to make groups of students with similar/homogeneous knowledge, different/heterogeneous knowledge, or random groups, based on the previous performance of the students on the course. Finally, the last option is “Data used to create the groups”, where the teacher can select the previous data (if any) that will be consider when creating homogeneous/heterogeneous groups, including videos watched by students or exercises tried, as indicated by Rubio-Fernández et al. (2019). Australasian Journal of Educational Technology, 2022, 38(6). 155 (a) (b) (c) Figure 2. Group creation: (a) create a new group or use predefined groups; (b) predefined groups; (c) group settings When the teacher decides to start the group work, the APP will recommend them some CLFPs (e.g., Jigsaw, Pyramid, Think-Pair-Share) (Figure 3a). The teacher can continue with the recommendation, select the default “Simple” pattern (one activity with the number of students and characteristics indicated by the teachers in the previous step), or select another pattern that they consider more suitable for the collaborative task. Once the teacher starts the groups, the screen will show all the groups that were created with their members (Figure 3b). (a) (b) Figure 3. Completion of group creation: (a) selection of CLFPs; (b) groups created with their members The teacher can select a group and right after will get three options: (a) “Change members”; (b) “Send link”; and (c) “Chat” (Figure 4a). If the teacher chooses the first option, the members of the group will be displayed on top and the members of another group will be displayed at the bottom (Figure 4b). The teacher can then switch students between the groups. The second option allows the teacher to send resources via links so that group members can access them from a web browser (Figure 4c). The first type of resource is “Document”. The system allows to select a document from the smartphone in which it is installed, Google Drive or from an external link and then a link is generated and sent to the specific group. The next type of resource is “Chat room”. The system allows for the creation of a new independent chat room to exchange Australasian Journal of Educational Technology, 2022, 38(6). 156 opinions with other groups or to talk after class. The third type of resource is “Tools”. The system allows the teacher to select complementary tools, such as Jamboard, Google Keeps, Google Calendar, other Google Suite tools or other tools via the URL, for collaboration. Finally, the last type of resource is “Others”. The system allows for the sending of any other URL that does not match the previous categories. The last option that can be seen in Figure 4a is a chat room shared between the teacher and the group (Figure 4d). (a) (b) (c) (d) Figure 4. Individual group: (a) the teachers' options with the individual group; (b) modifying groups; (c) sending resources; (d) chat room If the teacher selects all the groups (“All” button in Figure 3b), they would have the same three options as with an individual group (Figure 5a). In the case of the “Change members” option, the screen would be similar to that in Figure 4b but with no default group selected. The “Send Link” option allows the teacher to send resources to all students at the same time (Figure 5b). The “Chat” option allows the teacher to communicate with all students at the same time using a shared chat room (Figure 5c). (a) (b) (c) (d) Figure 5. All groups: (a) the teachers' options with all groups; (b) modifying groups; (c) sending resources; (d) chat room Student APP This application aims to allow the students to coordinate with their group mates and have access to the necessary resources to carry out the task or tasks assigned by the teacher. The APP also facilitates Australasian Journal of Educational Technology, 2022, 38(6). 157 communication between group members and the teacher. In addition, students present in class will be notified if they are not respecting the safety distance. The application requires the students to log in to their corporate account so that their data can be related to their assigned courses and classes (Figure 6a). After the students log in, they will be waiting to be assigned to groups (Figure 6b). In addition, Figure 6b also shows the notification of not respecting the safety distance on the top. This notification appears when the user is less than 1.5 metres away from another user for 10 seconds. (a) (b) Figure 6. First screens of Student APP: (a) login screen; (b) standby screen and notification of not respecting the safety distance When the students are assigned to groups they will automatically go to the next screen. This screen is different depending on whether the students are inside the classroom or online. Face-to-face students will see the group they belong to, and a map with their position and the place where they need to go to work with other group members that are also in the classroom (Figure 7a). This positioning also takes into account whether the safety distance is met so that students can work safely in teams. For online students, only the group they belong to will appear on screen (Figure 7b). a) b) Australasian Journal of Educational Technology, 2022, 38(6). 158 Figure 7. Indication of the group to which the student belongs: (a) screen for face-to-face students; (b) screen for online students The options available to students are shown at the bottom of the screens in Figure 7: (a) chat (group members can talk to each other, share resources and read the teacher's messages); (b) email (the students receive the link to the chat room to be able to connect from another device); and (c) resources (the students can directly access the resources provided by the teacher). These options are shown expanded in Figure 8. (a) (b) (c) Figure 8. Options available to the student: (a) chat room; (b) feedback on sending the URL; (c) available resources Architecture Smart Groups architecture is presented in Figure 9. Students and teachers access their respective applications from their smartphones. The applications connect to the system server using the university credentials. The system server uses these credentials to connect to the university servers and collect the necessary data from the teachers’ courses and classes, the students enrolled for these courses and classes, and the necessary data from these students (such as their previous performance in the course). When teachers create groups, the necessary calculations to suggest CLFPs are made on the system server. Smart Groups identifies the students who attend class (both physically or virtually) when logging in the APP. In addition, Smart Groups detects the students that are physically present in the classroom through beacons located in the classroom. When face-to-face students are assigned to groups, their position is calculated through the sensors of their smartphones and the WiFi connections with the routers located inside or near the classroom. Finally, Smart Groups can connect to the university tools, particularly those from Google Workspace (formerly G Suite for Education), such as Google Meet (for chat rooms), Google Drive for storing files, and Google Maps to get the campus map. The system also allows the use of other tools through URLs. Australasian Journal of Educational Technology, 2022, 38(6). 159 Figure 9. Architecture of Smart Groups Methodology An exploratory research study on Smart Groups was carried out. A mock-up was created to simulate and evaluate the behaviour of Smart Groups. After teachers and students had used the mock-up they were asked to complete a survey. Subsequently, some of the teachers were selected for individual interviews in order to discuss further the use of Smart Groups. The survey was intended to provide an overview and analysis of the usability of Smart Groups. The interviews were intended to go deeper and evaluate the usefulness of the system. The survey was sent to 60 users including teachers and students from different backgrounds. The objective was to have potential users to evaluate the usability of Smart Groups. The user survey had two parts. The first part collected demographic and background data to be able to define various profiles. The second part contained the system usability scale (SUS) (Brooke, 1996), a well-known instrument with 10 specific statements to be assessed on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree) to evaluate the usability of Smart Groups. SUS establishes a scale of values for the final result: less than or equal to 50 is not acceptable; between 51 and 69 is okay; between 70 and 84 is good; greater than or equal to 85 is excellent. The survey ended with three specific questions to evaluate the usefulness of Smart Groups with the same scale as SUS, and three open-ended questions to gather the general opinion of the users and to evaluate the strengths and weaknesses of the system. Appendix A contains the user survey used. Interviews were conducted with only a subset of 10 teachers who had completed the user survey. This subset was made up of teachers who were working or had worked with groups in hybrid or online learning environments. The aim of the interviews was to provide teachers with information that might not appear in the mock-up and to answer their questions. The interview consisted of the user using Teacher APP, while being informed of the options and data that could not be included in the mock-up. After the interviews, the teachers filled in a form of the technology acceptance model (TAM) of the perceived usefulness (Davis, 1989) for Smart Groups. This survey consisted of 14 Yes/No statements that explored different factors related to the usefulness of Smart Groups. Once the teachers answered the statements, the number of "Yes" Teacher Student s Teacher APP Student APP System Server University Smart Groups Other Tools URL Institutional Tools Classroom Beacon Router Australasian Journal of Educational Technology, 2022, 38(6). 160 answers to each statement was counted and divided by the number of teachers who conducted the interview. This resulted in a value between 0 and 1, with 1 the maximum score. Then, as in the previous survey, users had to rate the system on a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree) and answer the same three open questions of the previous survey. This was done to see if the explanations in the interview served to change the users’ opinions after completing the SUS survey. Appendix B contains the form that the user had to fill in. Results User survey Smart Groups was first evaluated by 60 users (41 students and 19 teachers) through the SUS survey. Table 1 presents the results considering all the users and the subgroups of teachers and students. The results for all users were: mean 75.54, standard deviation 14.88 and median 76.25. This score, according to Brooke (2013), can be classified as good. The results in the case of students were very similar: mean 75.18, standard deviation 16.88 and median 77.5. This score is also classified as good. The results in the case of teachers (including the 12 teachers and 7 users with both roles) were slightly better: mean 76.32, standard deviation 9.59 and median 75. This score is also classified as good. Table 1 Results of SUS survey according to user profile User type Mean Standard Deviation Median All (N = 60) 75.54 14.88 76.25 Students (N = 41) 75.18 16.88 77.5 Teachers (N = 19) 76.32 9.59 75 Overall, the statement, “I think the system is good for creating groups from the teacher's point of view”, obtained a mean value of 4.22 (standard deviation 0.87, median 4). The results in the case of teachers were also slightly higher: mean 4.32, standard deviation 0.67, and median 4. The statement “I think the system is good for coordination and teamwork from the student's point of view”, obtained a mean value of 3.87 (standard deviation 0.96, median 4). The results in the case of students were slightly higher: mean 3.78, standard deviation 0.95 and median 4. In addition, the overall score of Smart Groups was mean 3.95, standard deviation 0.7 and median 4. Teachers scored the system with a mean value of 4.00, standard deviation 0.58 and median 4. The students scored the system with a mean value of 3.93, standard deviation 0.75 and median 4. Finally, in the open-ended questions, users highlighted the following positive aspects of Smart Groups: the easiness to create groups in hybrid learning environment (6 students and 5 teachers), the ease of applying CLFPs (2 students and 6 teachers), the convenience of being able to work from a mobile phone (6 students and 2 teachers), the control of the safety distance (5 students and 3 teachers), and its overall usefulness (9 students and 9 teachers). In contrast, the disadvantages highlighted by the users were: the learning curve to get used to the system (6 students and 5 teachers), the need to see the system in a real scenario to be able to evaluate it better (2 students and 7 teachers), and the need for improving its visual appearance (5 students and 3 teachers). The users were asked what they would improve about the system and the most common answer was the possibility for students to share the results of their task with the teacher. Interviews Interviews were carried out with 10 of the teachers who completed the SUS survey. These teachers completed the TAM as part of their interview (Davis, 1989). Table 2 shows the characteristics referred to in each statement of the TAM. The results of the TAM range from 0 (all teachers marking “No”) to 1 (all teachers marking “Yes”) for each characteristic. In this case, the characteristic "Addresses my needs", which is relating to the usefulness of Smart Groups, was the most positively assessed, obtaining the value 1. In contrast, the most negatively assessed characteristic by the teachers was "Job difficult without". Australasian Journal of Educational Technology, 2022, 38(6). 161 Table 2 Results of the TAM form Item Factor analysis of perceived usefulness Job difficult without 0.5 Control over work 0.8 Job performance 0.7 Addresses my needs 1 Saves me time 0.9 Work more quickly 0.9 Critical to my job 0.7 Accomplish more work 0.7 Cut unproductive time 0.9 Effectiveness 0.7 Quality of work 0.6 Increase productivity 0.7 Makes job easier 0.9 Useful 1 The subgroup of teachers who were interviewed assessed Smart Groups with a mean value of 3.9 (standard deviation 0.57 and median 4). Finally, users highlighted the same positive aspects in the open questions as in the previous survey. The only additional disadvantage which was mentioned by the teachers was the lack of a more automatic creation of resources (2 users). When teachers were asked about what they would improve in Smart Groups, some additional ideas included: additional support to better understand CLFPs (4 users), the possibility to add already created groups (e.g., uploading an Excel/CSV file) (3 users), visualisations on students’ activity (e.g., use of exercises, connection times) (3 users), the automatic creation of resources (3 users), and the possibility to specify if groups are homogeneous or heterogeneous considering face-to-face students only or online students only (2 users). Discussion RQ1: Is Smart Groups a usable system to support collaboration in hybrid learning environments? The results obtained from SUS indicate that the usability of the system was good, and the overall feedback received was very positive. From these results, it can be seen that the group creation feature was what most attracted the users' attention. The most positive aspects of group creation, as indicated by users, were the ease of creation of groups, incorporating CLFPs. The fact that users perceived the overall use of the system as easy is positive since the comfort of students and teachers in collaborative activities taking place in hybrid learning environments is of importance as specified by previous studies (Bower et al., 2015; Stewart et al., 2011). Moreover, the positive perception of the inclusion of CLFPs in the design of collaborative activities thanks to Smart Groups was in line with the recommendations from several authors to facilitate the orchestration of collaborative activities (Hernández Leo, Asensio-Pérez, Dimitriadis, Bote-Lorenzo et al., 2005; Villasclaras-Fernández, et al. 2013). Similarities can also be seen with the works by Alvarez et al. (2019) and Hernández-Leo et al. (2006), where users also perceived positively the integration and management of CLFPs with the tools presented in those articles. Another feature that was highly rated by users was the warning about not complying with the safety distance, which users perceived as useful and innovative. This aligns with Wanga et al. (2020) and Ohme et al. (2020), who highlighted the importance of smartphones in keeping the safety distance in times of the COVID-19 pandemic. In conclusion, users perceived that the usability of Smart Groups was good, although they indicated the need to improve the information and support provided to the user when performing the task. RQ2: Is Smart Group useful to orchestrate collaboration in hybrid learning environments? The results of the TAM showed that users' perception of the usefulness of Smart Group was high. All 10 teachers agreed that the system met their needs and was useful. Moreover, 9 out of 10 teachers responded Australasian Journal of Educational Technology, 2022, 38(6). 162 that Smart Groups saved them time, allowed them to do their work faster, shortening inactivity time, and made their work easier. These are positive results, considering the typical problems that slow down the orchestration of collaborative tasks (Dimitriadis, 2012; Dimitriadis et al., 2013). Another important feature highlighted by the teachers (8 out of 10) was that Smart Groups gave them more control over their work. These results are similar to those obtained by Lee and Salman (2012) and Ochoa et al. (2009), who highlighted the usefulness of the systems they developed in saving time on collaborative tasks and accessing resources quickly. In contrast, there is room for improvement in Smart Groups, particularly in relation to the automatic creation of resources for all groups. This was stressed by the interviewees. All these data suggest that Smart Groups was perceived as useful for the orchestration of groups in hybrid learning environments, but still needs to be further adapted to the needs of its potential users. Conclusions and future work This paper has presented Smart Groups, a system designed to facilitate the orchestration of collaborative learning tasks, particularly in synchronous hybrid learning environments, with the aim to turn these into smart hybrid learning environments. Smart Groups facilitates group creation and management including CLFPs, considers the students who are in the classroom and those who are online, facilitates group coordination and communication among students, and helps maintain the necessary safety distance in the case of the teacher and students who are in the classroom. Smart Groups has been evaluated with SUS and TAM to assess its usability and usefulness respectively. This evaluation showed good results in both aspects, obtaining a good on the SUS scale and high usefulness perception factors on most of the TAM items. These results are in line with previous studies that also stressed the importance of using CLFPs to support collaborative learning, although further research is needed on their use to support synchronous hybrid learning environments. Furthermore, previous articles have shown good results in the use of tools to support synchronous hybrid learning environments but, as detected here and in previous articles, these tools need to be able to adapt to more diverse scenarios. This research covered orchestration in collaborative learning, hybrid learning scenarios, and the need to keep the safety distance due to COVID-19. The contribution of this research is the use of Smart Groups as a simulation study, focusing on its usability and usefulness. Some recommendations for practitioners in relation to the results obtained in this research include: (1) the need for further research on hybrid learning environments that combine onsite and online learners attending synchronous lectures; (2) the use of tools such as Smart Groups that facilitate the orchestration of collaboration in these hybrid learning environments; and (3) the use of tools such as Smart Groups that facilitate the maintenance of the safety distance in closed environments, such as the classroom, where the teacher wants to promote group collaboration. These results may inform for future research to address the challenge of collaborative learning in synchronous hybrid learning environments or for anyone looking for safety distance solutions when collaborating in groups. However, as an exploratory study, this research has several limitations. First is the use of a mock-up of Smart Groups for the evaluation. Although mock-ups are useful in the design phase as part of agile development methodologies, it is necessary to implement Smart Groups in order to evaluate it in real scenarios. This would allow a closer assessment of the impact Smart Groups would have on collaborative learning in hybrid learning environments. Of the 60 users who participated in the user survey, 10 of them indicated that it was difficult for them to understand the scope of the system due to constraints derived from the use of the mock-up. The second limitation is the number of users who participated in the evaluation. Only 60 users completed the user survey and of these only 10 (all in the role of teacher) participated in the interviews and completed the TAM. Further these teachers had extensive experience in education and some were working in hybrid learning environments. Smart Groups needs to be evaluated by more users. This would allow comparing more opinions and increase the validity of the evaluation conducted here. Finally, it is necessary to consider the users’ needs identified in this evaluation: (1) the provision of more information and support on the use of CLFPs; and (2) the possibility to create resources automatically for all groups, instead of group by group. Acknowledgements This work was supported in part by the FEDER/Ministerio de Ciencia, Innovación y Universidades– Agencia Estatal de Investigación, through the Smartlet Project under Grant TIN2017-85179-C3-1-R, and Australasian Journal of Educational Technology, 2022, 38(6). 163 in part by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS- 4307 and under the Multiannual Agreement with UC3M in the line of Excellence of University Professors (EPUC3M21), and in the context of the V PRICIT (Regional Programme of Research and Technological Innovation), a project which is co-funded by the European Structural Funds (FSE and FEDER). Partial support has also been received from the European Commission through Erasmus+ Capacity Building in the Field of Higher Education projects, more specifically through projects LALA, InnovaT and PROF-XXI (586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP), (598758-EPP-1-2018-1-AT-EPPKA2-CBHE-JP), (609767-EPP-1-2019-1-ES-EPPKA2-CBHE-JP). This publication reflects the views only of the authors and funders cannot be held responsible for any use which may be made of the information contained therein. References Adams, F. H. (2013). Using jigsaw technique as an effective way of promoting cooperative learning among primary six pupils in Fijai. International Journal of Education and Practice, 1(6), 64-74. https://doi.org/10.18488/journal.61/2013.1.6/61.6.64.74 Al Lily, A. E., Ismail, A. F., Abunasser, F. M., & Alhajhoj Alqahtani, R. H. (2020). Distance education as a response to pandemics: Coronavirus and Arab culture. Technology in Society, 63(101317), 1-11. https://doi.org/10.1016/j.techsoc.2020.101317 Alvarez C., Zurita G., Baloian N., Jerez O., Peñafiel S. (2019). A CSCL script for supporting moral reasoning in the ethics classroom. In H. Nakanishi, H. Egi, I. A. Chounta, H. Takada, S. Ichimura, & U. Hoppe (Eds.), Collaboration technologies and social computing. CRIWG+CollabTech 2019. Lecture notes in computer science (Vol 11677). Springer. https://doi.org/10.1007/978-3-030-28011- 6_5 Azlina, N. N. (2010). CETLs: Supporting collaborative activities among students and teachers through the use of Think-Pair-Share techniques. International Journal of Computer Science Issues, 7(5), 1-18. http://www.ijcsi.org/papers/7-5-18-29.pdf Azukas, M. E. (2019). Cultivating blended communities of practice to promote personalized learning. Journal of Online Learning Research, 5(3), 251-274. https://www.learntechlib.org/primary/p/210640/ Bae, M. Y., & Cho, D. J. (2015). Design and implementation of automatic attendance check system using BLE beacon. International Journal of Multimedia and Ubiquitous Engineering, 10(10), 177-186. https://www.earticle.net/Article/A257270 Bayğın, M., Yaman, O., Topuz, A., & Kaleli, S. (2021). RFID based authorization method for computer systems in Smart Library environments. Balkan Journal of Electrical and Computer Engineering, 9(1), 33-39. https://doi.org/10.17694/bajece.710051 Bdiwi, R., de Runz, C., Faiz, S., & Cherif, A. A. (2019). Smart learning environment: Teacher's role in assessing classroom attention. Research in Learning Technology, 27(2072), 1-14. https://eric.ed.gov/?id=EJ1203430 Bower, M., Dalgarno, B., Kennedy, G. E., Lee, M. J., & Kenney, J. (2015). Design and implementation factors in blended synchronous learning environments: Outcomes from a cross-case analysis. Computers & Education, 86, 1-17. https://doi.org/10.1016/j.compedu.2015.03.006 Brooke, J. (1996). SUS: SUS: a ‘quick and dirty’ usability. Usability evaluation in industry. Taylor & Francis. https://www.taylorfrancis.com/chapters/edit/10.1201/9781498710411-24/quick-dirty- usability-tests-bruce-thomas Brooke, J. (2013). SUS: A retrospective. Journal of Usability Studies, 8(2), 29-40. http://uxpajournal.org/wp-content/uploads/sites/8/pdf/JUS_Brooke_February_2013.pdf Carruana Martín, A., Alario-Hoyos, C., & Delgado Kloos, C. (2019). Smart education: A review and future research directions. 31(57), 1-10. https://doi.org/10.3390/proceedings2019031057 Chang, C. L., & Fang, M. (2020). E-learning and online instructions of higher education during the 2019 novel coronavirus diseases (COVID-19) epidemic. Journal of Physics: Conference Series, 1574, 1-6. https://doi.org/10.1088/1742-6596/1574/1/012166 Childress, M. D., & Braswell, R. (2006). Using massively multiplayer online role‐playing games for online learning. Distance Education, 27(2), 187-196. https://doi.org/10.1080/01587910600789522 Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340. https://doi.org/10.2307/249008 Dillenbourg, P. (1999). What do you mean by collaborative learning? Collaborative learning: Cognitive and computational approaches. Elsevier. https://telearn.archives-ouvertes.fr/hal-00190240 Dillenbourg, P., & Jermann, P. (2010). Technology for classroom orchestration. New science of learning. Springer. https://doi.org/10.1007/978-1-4419-5716-0_26 https://doi.org/10.18488/journal.61/2013.1.6/61.6.64.74 https://doi.org/10.1016/j.techsoc.2020.101317 https://doi.org/10.1007/978-3-030-28011-6_5 https://doi.org/10.1007/978-3-030-28011-6_5 http://www.ijcsi.org/papers/7-5-18-29.pdf https://www.learntechlib.org/primary/p/210640/ https://www.earticle.net/Article/A257270 https://doi.org/10.17694/bajece.710051 https://eric.ed.gov/?id=EJ1203430 https://doi.org/10.1016/j.compedu.2015.03.006 http://uxpajournal.org/wp-content/uploads/sites/8/pdf/JUS_Brooke_February_2013.pdf https://doi.org/10.3390/proceedings2019031057 https://doi.org/10.1088/1742-6596/1574/1/012166 https://doi.org/10.1080/01587910600789522 https://doi.org/10.2307/249008 https://telearn.archives-ouvertes.fr/hal-00190240 https://doi.org/10.1007/978-1-4419-5716-0_26 Australasian Journal of Educational Technology, 2022, 38(6). 164 Dillenbourg, P., Nussbaum, M., Dimitriadis, Y., & Roschelle, J. (2013). Design for classroom orchestration. Computers & Education, 69, 485-492. http://dx.doi.org/10.1016/j.compedu.2013.04.013 Dimitriadis Y. A. (2012) Supporting teachers in orchestrating CSCL classrooms. In A. Jimoyiannis (Ed.), Research on e-learning and ICT in education (pp. 71-82). Springer. https://doi.org/10.1007/978-1- 4614-1083-6_6 Dimitriadis, Y. A., Prieto, L. P., & Asensio-Pérez, J. I. (2013). The role of design and enactment patterns in orchestration: Helping to integrate technology in blended classroom ecosystems. Computers & Education, 69, 496-499. https://doi.org/10.1016/j.compedu.2013.04.004 Gokhale, A. A. (1995). Collaborative learning enhances critical thinking. Journal of Technology Education, 7(1), 22-30. https://doi.org/10.21061/jte.v7i1.a.2 Guo, J., Liu, X., & Wang, Z. (2015, August). Optimized indoor positioning based on WIFI in mobile classroom project. Proceedings of the 11th International Conference on Natural Computation, (pp. 1208-1212). https://doi.org/10.1109/ICNC.2015.7378163 Hämäläinen, R., & Oksanen, K. (2012). Challenge of supporting vocational learning: Empowering collaboration in a scripted 3D game–How does teachers’ real-time orchestration make a difference? Computers & Education, 59(2), 281-293. https://doi.org/10.1016/j.compedu.2012.01.002 Hartono, S., Kosala, R., Supangkat, S. H., & Ranti, B. (2018, October 10-11). Smart hybrid learning framework based on three-layer architecture to bolster up Education 4.0 [Paper presentation]. International Conference on ICT for Smart Society (pp. 1-5). https://doi.org/10.1109/ICTSS.2018.8550028. Hauge, J. B., Stefan, I. A., Stefan, A., Cazzaniga, M., Yanez, P., Skupinski, T., & Mohier, F. (2017, December). Exploring context-aware activities to enhance the learning experience. Proceedings of International Conference on Games and Learning Alliance (pp. 238-247). Springer. https://doi.org/10.1007/978-3-319-71940-5_22 Hernández-Leo D., Asensio-Pérez J. I., Derntl M., Prieto L. P., Chacón J. (2014) ILDE: Community environment for conceptualizing, authoring and deploying learning activities. In C. Rensing, S. de Freitas, T. Ley, & P. J. Muñoz-Merino (Eds.), Open learning and teaching in educational communities. Lecture notes in computer science (Vol. 8719, pp. 490-493). Springer. https://doi.org/10.1007/978-3-319-11200-8_48 Hernández Leo, D., Asensio-Pérez, J. I., & Dimitriadis, Y. (2005). Computational representation of collaborative learning flow patterns using IMS learning design. Journal of Educational Technology & Society, 8(4), 75-89. https://www.jstor.org/stable/10.2307/jeductechsoci.8.4.75 Hernández Leo, D., Asensio-Pérez, J. I., Dimitriadis, Y., Bote-Lorenzo, M. L., Jorrín-Abellán, I. M., & Villasclaras-Fernández, E. D. (2005). Reusing IMS-LD formalized best practices in collaborative learning structuring. Advanced Technology for Learning, 2(3): 223-32. https://doi.org/10.2316/Journal.208.2005.4.208-0865 Hernández-Leo, D., Villasclaras-Fernández, E. D., Asensio-Pérez, J. I., Dimitriadis, Y., Jorrín-Abellán, I. M., Ruiz-Requies, I., & Rubia-Avi, B. (2006). COLLAGE: A collaborative learning design editor based on patterns. Journal of Educational Technology & Society, 9(1), 58-71. http://www.jstor.org/stable/jeductechsoci.9.1.58 Herrera-Pavo, M. Á. (2021). Collaborative learning for virtual higher education. Learning, Culture and Social Interaction, 28 1-11. https://doi.org/10.1016/j.lcsi.2020.100437 Hrastinski, S. (2008). Asynchronous and synchronous e-learning. Educause quarterly, 31(4), 51-55. https://er.educause.edu/articles/2008/11/asynchronous-and-synchronous-elearning Hwang, G. (2014). Definition, framework and research issues of smart learning environments - a context- aware ubiquitous learning perspective. Smart Learning Environments, 1(4), 1-14. https://doi.org/10.1186/s40561-014-0004-5 Jamison, A., Kolmos, A., & Holgaard, J. E. (2014). Hybrid learning: An integrative approach to engineering education. Journal of Engineering Education, 103(2), 253-273. https://doi.org/10.1002/jee.20041 Lee, K. B., & Salman, R. (2012). The design and development of mobile collaborative learning application using android. Journal of Information Technology and Application in Education, 1(1), 1-8. http://www.airitilibrary.com/Publication/alDetailedMesh?DocID=P20150611010-201203- 201507310014-201507310014-1-8 Manathunga, K., & Hernández‐Leo, D. (2018). Authoring and enactment of mobile pyramid‐based collaborative learning activities. British Journal of Educational Technology, 49(2), 262-275. https://doi.org/10.1111/bjet.12588 http://dx.doi.org/10.1016/j.compedu.2013.04.013 https://doi.org/10.1007/978-1-4614-1083-6_6 https://doi.org/10.1007/978-1-4614-1083-6_6 https://doi.org/10.1016/j.compedu.2013.04.004 https://doi.org/10.21061/jte.v7i1.a.2 https://doi.org/10.1109/ICNC.2015.7378163 https://doi.org/10.1016/j.compedu.2012.01.002 https://doi.org/10.1109/ICTSS.2018.8550028 https://doi.org/10.1007/978-3-319-71940-5_22 https://doi.org/10.1007/978-3-319-11200-8_48 https://www.jstor.org/stable/10.2307/jeductechsoci.8.4.75 https://doi.org/10.2316/Journal.208.2005.4.208-0865 http://www.jstor.org/stable/jeductechsoci.9.1.58 https://doi.org/10.1016/j.lcsi.2020.100437 https://er.educause.edu/articles/2008/11/asynchronous-and-synchronous-elearning https://doi.org/10.1186/s40561-014-0004-5 https://doi.org/10.1002/jee.20041 http://www.airitilibrary.com/Publication/alDetailedMesh?DocID=P20150611010-201203-201507310014-201507310014-1-8 http://www.airitilibrary.com/Publication/alDetailedMesh?DocID=P20150611010-201203-201507310014-201507310014-1-8 https://doi.org/10.1111/bjet.12588 Australasian Journal of Educational Technology, 2022, 38(6). 165 Martinez-Maldonado, R. (2019, March). "I spent more time with that team": Making spatial pedagogy visible using positioning sensors. Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19). Association for Computing Machinery, New York, NY. https://doi.org/10.1145/3303772.3303818 Mittal, Y., Varshney, A., Aggarwal, P., Matani, K., & Mittal, V. K. (2015, December 17-20). Fingerprint biometric based access control and classroom attendance management system. Paper presented at the 2015 Annual IEEE India Conference, New Delhi, India. https://doi.org/10.1109/INDICON.2015.7443699 Nieswandt, M., McEneaney, E. H., & Affolter, R. (2020). A framework for exploring small group learning in high school science classrooms: The triple problem solving space. Instructional Science, 48(3), 243-290. https://doi.org/10.1007/s11251-020-09510-9 Ochoa, S., Alarcon, R., & Guerrero, L. (2009, July). Understanding the relationship between requirements and context elements in mobile collaboration. Proceedings of International Conference on Human- Computer Interaction (pp. 67-76). Springer. https://doi.org/10.1007/978-3-642-02580-8_8 Ohme, J., Abeele, M. M. V., Van Gaeveren, K., Durnez, W., & De Marez, L. (2020). Staying informed and bridging “social distance”: Smartphone news use and mobile messaging behaviors of Flemish adults during the first weeks of the COVID-19 pandemic. Socius, 6. https://doi.org/10.1177%2F2378023120950190 Olapiriyakul, K., & Scher, J. M. (2006). A guide to establishing hybrid learning courses: Employing information technology to create a new learning experience, and a case study. The Internet and Higher Education, 9(4), 287-301. https://doi.org/10.1016/j.iheduc.2006.08.001 Olsen, J. K., Rummel, N., & Aleven, V. (2020). Designing for the co-orchestration of social transitions between individual, small-group and whole-class learning in the classroom. International Journal of Artificial Intelligence in Education, 31(1), 24-56. https://doi.org/10.1007/s40593-020-00228-w Pérez-Sanagustín, M., Hilliger, I., Alario-Hoyos, C., Kloos, C. D., & Rayyan, S. (2017). H-MOOC framework: Reusing MOOCs for hybrid education. Journal of Computing in Higher Education, 29(1), 47-64. https://doi.org/10.1007/s12528-017-9133-5 Prieto, L. P., Dimitriadis, Y., Asensio-Pérez, J. I., & Looi, C. K. (2015). Orchestration in learning technology research: Evaluation of a conceptual framework. Research in Learning Technology, 23, 1- 15. https://doi.org/10.3402/rlt.v23.28019 Prieto, L. P., Holenko Dlab, M., Gutiérrez, I., Abdulwahed, M., & Balid, W. (2011). Orchestrating technology enhanced learning: A literature review and a conceptual framework. International Journal of Technology Enhanced Learning, 3(6), 583-598. https://doi.org/10.1504/IJTEL.2011.045449 Raes, A., Detienne, L., Windey, I., & Depaepe, F. (2020). A systematic literature review on synchronous hybrid learning: gaps identified. Learning Environments Research, 23(3), 269-290. https://doi.org/10.1007/s10984-019-09303-z Roschelle, J., Dimitriadis, Y., & Hoppe, U. (2013). Classroom orchestration: Synthesis. Computers & Education, 69, 523-526. https://doi.org/10.1016/j.compedu.2013.04.010 Roseth, C., Akcaoglu, M., & Zellner, A. (2013). Blending synchronous face-to-face and computer- supported cooperative learning in a hybrid doctoral seminar. TechTrends, 57(3), 54-59. https://doi.org/10.1007/s11528-013-0663-z Rubio-Fernández, A., Muñoz-Merino, P. J., & Delgado Kloos, C. (2019, June). Analyzing the group formation process in intelligent tutoring systems. Proceedings of International Conference on Intelligent Tutoring Systems (pp. 34-39). Springer. https://doi.org/10.1007/978-3-030-22244-4_5 Stewart, A. R., Harlow, D. B., & DeBacco, K. (2011). Students’ experience of synchronous learning in distributed environments. Distance Education, 32(3), 357-381. https://doi.org/10.1080/01587919.2011.610289 Subedi, S., Kwon, G. R., Shin, S., Hwang, S. S., & Pyun, J. Y. (2016, July). Beacon based indoor positioning system using weighted centroid localization approach. Proceedings of Eighth International Conference on Ubiquitous and Future Networks (pp. 1016-1019). https://doi.org/10.1109/ICUFN.2016.7536951 Sun, C., & Zhai, Z. (2020). The efficacy of social distance and ventilation effectiveness in preventing COVID-19 transmission. Sustainable Cities and Society, 62, 102390. https://doi.org/10.1016/j.scs.2020.102390 Tabuenca, B., Serrano-Iglesias, S., Carruana Martín, A., Villa-Torrano, C., Dimitriadis, Y., Asensio- Perez, J. I., Alario-Hoyos, C., Gomez-Sanchez, E., Bote-Lorenzo, M.L., Martınez-Monés, A., & Delgado Kloos, C. (2020). Affordances and core functions of smart learning environments: A systematic literature review. Manuscript submitted for publication. https://doi.org/10.1145/3303772.3303818 https://doi.org/10.1109/INDICON.2015.7443699 https://doi.org/10.1007/s11251-020-09510-9 https://doi.org/10.1007/978-3-642-02580-8_8 https://doi.org/10.1177%2F2378023120950190 https://doi.org/10.1016/j.iheduc.2006.08.001 https://doi.org/10.1007/s40593-020-00228-w https://doi.org/10.1007/s12528-017-9133-5 https://doi.org/10.3402/rlt.v23.28019 https://doi.org/10.1504/IJTEL.2011.045449 https://doi.org/10.1007/s10984-019-09303-z https://doi.org/10.1016/j.compedu.2013.04.010 https://doi.org/10.1007/s11528-013-0663-z https://doi.org/10.1007/978-3-030-22244-4_5 https://doi.org/10.1080/01587919.2011.610289 https://doi.org/10.1109/ICUFN.2016.7536951 https://doi.org/10.1016/j.scs.2020.102390 Australasian Journal of Educational Technology, 2022, 38(6). 166 van der Laan Smith, J., & Spindle, R. M. (2007). The impact of group formation in a cooperative learning environment. Journal of Accounting Education, 25(4), 153-167. https://doi.org/10.1016/j.jaccedu.2007.09.002 Villasclaras-Fernández, E., Hernández-Leo, D., Asensio-Pérez, J. I., & Dimitriadis, Y. (2013). Web collage: An implementation of support for assessment design in CSCL macro-scripts. Computers & Education, 67, 79-97. https://doi.org/10.1016/j.compedu.2013.03.002 Wanga, H., Joseph, T., & Chuma, M. B. (2020). Social distancing: Role of smartphone during coronavirus (COVID–19) pandemic era. International Journal of Computer Science and Mobile Computing, 9(5), 181-188. https://ijcsmc.com/docs/papers/May2020/V9I5202016.pdf Williamson, B., Eynon, R., & Potter, J. (2020). Pandemic politics, pedagogies and practices: Digital technologies and distance education during the coronavirus emergency. Learning, Media and Technology, 45(2), 107-114. https://doi.org/10.1080/17439884.2020.1761641 Zorić, B., Dudjak, M., Bajer, D., & Martinović, G. (2019, May). Design and development of a smart attendance management system with Bluetooth low energy beacons. Proceedings of Zooming Innovation in Consumer Technologies Conference, (pp. 86-91). https://doi.org/10.1109/ZINC.2019.8769433 Corresponding author: Adrián Carruana Martín, acarruan@inf.uc3m.es Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC- ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY- NC-ND 4.0. Please cite as: Carruana Martín, A., & Alario-Hoyos, C., & Delgado Kloos, C. (2022). Smart Groups: A system to orchestrate collaboration in hybrid learning environments. A simulation study. Australasian Journal of Educational Technology, 38(6), 150-168. https://doi.org/10.14742/ajet.6776 https://doi.org/10.1016/j.jaccedu.2007.09.002 https://doi.org/10.1016/j.compedu.2013.03.002 https://ijcsmc.com/docs/papers/May2020/V9I5202016.pdf https://doi.org/10.1080/17439884.2020.1761641 https://doi.org/10.1109/ZINC.2019.8769433 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.6776 Australasian Journal of Educational Technology, 2022, 38(6). 167 Appendix A SUS survey This evaluation consists of a SUS survey with 10 specific statements to be assessed in a scale from 1 (strongly disagree) to 5 (strongly agree). Statement Strongly disagree Strongly agree I think that I would like to use this system frequently 1 2 3 4 5 I found the system unnecessarily complex 1 2 3 4 5 I thought the system was easy to use 1 2 3 4 5 I think that I would need the support of a technical person to be able to use this system 1 2 3 4 5 I found the various functions in the system were well integrated 1 2 3 4 5 I thought there was too much inconsistency in this system 1 2 3 4 5 I would imagine that most people would learn to use this system very quickly 1 2 3 4 5 I found the system very awkward to use 1 2 3 4 5 I felt very confident using the system 1 2 3 4 5 I needed to learn a lot of things before I could get going with this system 1 2 3 4 5 Also, two additional statements and one question were incorporated to collect more information. Statement Strongly disagree Strongly agree I think the system is good for creating groups from the teacher's point of view 1 2 3 4 5 I think the system is good for coordination and teamwork from the student's point of view 1 2 3 4 5 How would I rate the system? (1 minimum value, 5 maximum value) 1 2 3 4 5 Three additional open-ended questions were added with the aim to gather the general opinion of the users and to identify the strengths and weaknesses of the system: • What are the main difficulties you have encountered in the system? • What are the main virtues you have found in the system? • What would you improve about the system? Australasian Journal of Educational Technology, 2022, 38(6). 168 Appendix B TAM survey This evaluation consists of a TAM survey of 14 Yes or No questions that explore different factors related to the usefulness of the system. Statement Yes No My work would be difficult to do without Smart Groups Using Smart Groups gives me more control over my work Using Smart Groups improves my work performance Smart Groups meets the needs of my work Using Smart Groups saves me time Smart Groups allow me to perform tasks more quickly Smart Groups supports the critical aspects of my work Using Smart Groups allows me to do more work than would otherwise be possible The use of Smart Groups reduces the time I spend on unproductive activities Using Smart Groups increases my efficiency at work Using Smart Groups improves the quality of the work I do Using Smart Groups increases my productivity Using Smart Groups makes my job easier In general, I find Smart Groups useful in my work Also, an additional question was incorporated to collect more information. Statement Strongly disagree Strongly agree How would I rate the system? 1 2 3 4 5 Three additional open-ended questions were added with the aim to gather the general opinion of the users and to identify the strengths and weaknesses of the system: • What are the main difficulties you have encountered in the system? • What are the main virtues you have found in the system? • What would you improve about the system? Introduction Research problem and objectives Related work Theoretical framework Collaboration and hybrid learning environments Positioning in the classroom Design Teacher APP Student APP Architecture Methodology Results User survey Interviews Discussion RQ1: Is Smart Groups a usable system to support collaboration in hybrid learning environments? RQ2: Is Smart Group useful to orchestrate collaboration in hybrid learning environments? Conclusions and future work Acknowledgements References Appendix A SUS survey Appendix B TAM survey