Australasian Journal of Educational Technology, 2020, 36(4). 57 The effects of a visible-annotation tool for sequential knowledge construction on discourse patterns and collaborative outcomes Yoonhee Shin School of Liberal Art, Dankook University, Gyeonggi, South Korea Jaewon Jung Korean Educational Development Institute (KEDI), Chungbuk, South Korea This study aimed to explore learners’ discourse patterns and outcomes while using a visible-annotation tool as a collaborative representation tool. The tool used in this study introduced two types of sharing activities before the problem-solving phase to support sequential knowledge construction. Forty participants were randomly assigned to one of three groups according to two variables: type of sharing activities (meaning sharing activity (M) and opinion sharing activity (O)), and type of representation function to guide sharing activities (word-based function (W) and sentence-based function (S)). All three groups performed sharing activities during the same period. After completing these, the participants carried out a lesson-planning task in pairs during the problem-solving phase. All annotations across three learning phases were categorised to investigate discourse patterns. The findings revealed that Group MWOS, provided with M based on W and O based on S, had the most effective knowledge construction process, showing sequential discourse patterns. In addition, differences in discourse patterns among groups positively influenced the level of collaborative outcomes. Implications for practice or policy: • University students could perform invaluable interactions and achieve learning outcomes in complex learning by receiving sequential part-tasks from simple to complex. • Course leaders may need to consider instructional strategies for sequential knowledge construction in complex learning to promote meaningful interactions. Keywords: computer-supported collaborative learning (CSCL), representation tool, knowledge construction, discourse patterns, collaborative outcomes Introduction Collaborative learning is an effective learning method for enabling individuals to share ideas and build knowledge through active interactions (Frensen, Weinberger, & Kirschner, 2013). Approaching issues to be solved together and sharing opinions within a team can help reduce misunderstandings. Collaboration can also help to correct, supplement, and deepen discussion (Roschelle & Teasley, 1995). However, learners often have difficulty accurately understanding each other’s thinking during the collaboration and communication process (Barron, 2003; Dillenbourg & Betrancourt, 2006). Studies on collaborative learning have thus tried to overcome this difficulty using computer-supported collaborative learning (CSCL) tools that aim to support and enable effective interaction, guiding students to actively participate in knowledge construction activities (Ludvigsen, 2016). CSCL tools can reduce unnecessary collaboration load and generate various kinds of interaction by providing tool functions and direct guidance (Bruggen, Kirschner, & Jochems, 2002). Among CSCL tools, representation tools are effective for helping learners organise, express, and share knowledge or ideas using concept maps, threaded discussions, or linked annotations (Kolloffel, Eysink, & de Jong, 2011). However, several studies have pointed out that representation tools for CSCL lack consideration for the specific characteristics and complexity of learning tasks based on a collaborative knowledge construction process (Barron, 2003; Slof, Erkens, Kirschner, Janssen, & Jaspers, 2012). For example, some studies have suggested that concept maps and threaded discussions face limitations in reducing collaboration load or supporting complex learning, which requires a variety of sub-activities (Eryilmaz, Alrushiedat, Australasian Journal of Educational Technology, 2020, 36(4). 58 Kasemvilas, Mary, & Pol, 2009; Suthers & Hundhausen, 2003). Some studies have been conducted using a linked annotation tool that connects a specific part of the learning content with automatic annotations (Eryilmaz, van der Pol, Ryan, Clark, & Mary, 2013). Although the tool was able to increase various interactions by reducing collaborative load in the problem-solving phase, it could not induce interactions such as clarification and consensus building, which are required for building an accurate common ground among learners (Eryilmaz et al., 2013; Hull & Saxon, 2009). As a subsequent study on representation tools, Shin, Kim, and Jung (2018) proposed a visible-annotation tool (VAT) with linked annotations based on the process of collaborative knowledge construction. It provides two knowledge sharing phases with different tool functions supporting each phase before learners begin problem-solving, perform a meaning sharing activity with a word-based function, and complete an opinion sharing activity with a sentence-based function. The empirical findings of the study indicated that VAT has a significant effect on the accuracy of shared knowledge and the level of collaborative performance (Shin et al., 2018). However, collaborative processes such as discourse patterns or cognitive load aspects have been insufficiently explored to more precisely identify these correlations. In collaborative learning processes, an increase in the number of valueless interactions may impede high- level learning performance and limit the quality of discussion activities (Beers, Boshuizen, Kirschner, & Gijselaers, 2005). In order to reduce meaningless interactions and induce more meaningful exchanges that are beneficial for learning, it is necessary to provide learners with opportunities to clarify and interpret learning content, in addition to presenting questions, opinions, and conflicts to ultimately derive a solution through consensus building and agreement (Beers et al., 2005; Pena-Shaff & Nicholls, 2004). Slof, Erkens, Kirschner, Janssen, and Jaspers (2012) suggested providing instructional support that enables communication activities to analyse, discuss, and apply learning content in collaborative learning environments. However, most previous research has assessed the effectiveness of online collaboration tools using simple analyses based on the type of interaction, amount of comments, or collaborative performance. This level of information is not sufficient for investigating how types of interactions that occur during the collaboration process affect shared knowledge or constructed knowledge building. Janssen, Kirschner, Erkens, Kirschner, and Paas (2010) argued that more detailed studies on collaborative load are needed to gain real insight into optimising a CSCL environment, with careful analyses including consideration of a process-oriented approach (i.e., interaction patterns) as well as an effect-oriented approach (i.e., collaboration outcomes). Therefore, the purpose of this study was to derive practical implications for the design of representation tools and support collaborative knowledge construction. This study analysed the effect of the representation tool VAT on learners’ discourses and collaboration performance. The study focused on the following research question: What effect does managing strategies with VAT have on online discourse patterns, collaborative load, and collaborative learning outcomes? Specifically, there are two managing strategies for VAT: types of sharing activities (meaning sharing activity (M) and opinion sharing activity (O)), and types of representation functions to guide sharing activities (word-based function (W) and sentence-based function (S)). Literature review The importance of exploring the knowledge construction process in CSCL Much theoretical research has addressed the importance of CSCL environments from the perspective of learner interactions and knowledge construction (Cacciamani, Cesareni, Martini, Ferrini, & Fujita, 2012; Stahl, 2015). Specifically, Beers et al. (2005) stated that successful knowledge construction when facing a complex problem requires several types of interactions: contribution, verification, clarification, acceptance or rejection, and statements of position. These interaction types are essential for negotiating ideas or knowledge when solving a complex task, but different approaches are required according to the nature of each negotiation phase (i.e., clarification is required for meaning negotiation, while acceptance or rejection is for opinion negotiation) (Pena-Shaff & Nicholls, 2004; Slof et al., 2012). Australasian Journal of Educational Technology, 2020, 36(4). 59 Based on existing theoretical research, empirical studies have also been carried out to verify the relationship between knowledge construction quality and patterns of interaction (Fuerstenau, Ryssel, & Kunath, 2010). However, there is still a lack of work utilising process-oriented approaches for exploring collaborative cognition processes and learning outcomes as related to interaction patterns (Fu, van Aalst, & Chan, 2016; Yücel & Usluel, 2016). Nuanced differences exist among interaction activities and the repeated sharing phases used to address complex tasks, which necessitate research to analyse interaction processes more systematically (Wang, Anderson, Chen, & Barbera, 2017). Among previous empirical research, Erilymaz et al. (2013) did conduct a specific analysis of interactions, but the effect of the collaboration outcome remained ambiguous, and interaction types were limited to assertion and conflict. In addition, most research has had difficulty in exploring sequential learning processes, focusing only on analysing the frequency of interactions used in the problem-solving phase to derive a solution (Janssen et al., 2010). Collaborative load in CSCL Cognitive load theory (CLT) emphasises that managing limited cognitive capacity is indispensable for solving complex tasks (Dillenbourg & Betrancourt, 2006; Leppink, Paas, Van der Vleuten, Van Gog, & Van Merriënboer, 2013). CLT is concerned with instructional design strategies for optimising working memory loads to transfer acquired knowledge into long-term memory for successful problem-solving (Seufert & Brünken, 2006; van Merriënboer & Sweller, 2005). In particular, optimising working memory loads has become a central goal for CSCL, requiring complex tasks to be solved through several iterations of interaction activities (Ludvigsen, 2016). Most CSCL environments based on socio-constructivist roots include a problem-solving space and social interaction space (e.g., chat, forum, or discussion) (Dillenbourg & Betrancourt, 2006). The primary idea of CSCL is that meaningful interactions between learners can lead to the discovery of a solution by establishing common ground where unshared knowledge can be detected and negotiated (Slof et al., 2012). Although meaningful interaction with others can deepen students’ learning, cognitive load research has focused more on the additional effort required in interaction processes since collaborative load from engaging in interactive activities can foster unshared or unnecessary interactions through errors, negative conflicts, and duplication (Bernard & Lundgren-Cayrol, 2001). Even if active interactions for learning take place, some invaluable interactions without understanding of each other’s thinking may impede high-level learning outcomes and limit the quality of discussion activities. Therefore, research to optimise cognitive load for CSCL has suggested that an ideal CSCL environment should consider collaborative load as a crucial factor for inducing meaningful interaction by minimising unnecessary loads (i.e., task-related and tool-related loads) (Bruggen, Kirschner, & Jochems, 2002; Kirschner, Paas, & Kirschner, 2009). The effect of a representation tool on collaborative learning To manage collaborative load in CSCL, a variety of tools have been developed using representation functions to visualise learners’ cognitive process (e.g., concept map, graph, matrix, and annotation). Representative functions can be classified based on learning objectives: evidence maps for understanding and structuring knowledge (e.g., Engelmann & Hesse, 2010; Suthers & Hundhausen, 2003); linked boxes for constructing logical theories (e.g., Bell, 1997); and threaded discussions for sharing knowledge and opinions (e.g., Eryilmaz et al., 2013). These functions have been verified to induce learners’ attention to specific parts of the learning content (Kolloffel et al., 2011). In particular, using an asynchronous discussion tool for complex tasks is effective for representing learners’ cognitive processes systematically by providing them with enough time to consider others’ opinions (Schellens & Valcke, 2006). However, some studies have pointed out that a lack of systematic instructional design leads to a failure to achieve sequential knowledge construction based on dynamic interaction patterns from convergence to divergence (Beers et al., 2005; Suthers, Vatrapu, Medina, Joseph, & Dwyer, 2008). It has been claimed that most empirical research on designing tools has not considered the nature of complex tasks involving several kinds of part-tasks (Slof et al., 2012). Although effort has been made to find an optimal strategy for meaningful knowledge construction by managing cognitive load, both tasks and tool functions must be considered when designing a tool (Kolloffel et al., 201l; Nelson & Erlandson, 2008). For instance, Australasian Journal of Educational Technology, 2020, 36(4). 60 Fuerstenau et al. (2010) showed that a tool combining concept mapping and a summary writing function could efficiently promote learning results throughout the whole process. Slof et al. (2012) revealed the effectiveness of considering the nature of part-tasks to improve collaborative performance. However, a lack of focus on the initial phase of learning has impeded the description of sequential discourse patterns and verification of effects from cognitive load perspectives (Shin et al., 2018). To consider the construction of an exact common ground during meaningful interactions as the heart of the knowledge construction process (Beers et al., 2005; Seufert & Brünken, 2006), Shin et al. (2018) gave some attention to the initial learning phase to provide learners opportunities to share their thinking of learning content. They proposed a visible-annotation tool (VAT), which consisted of three learning phases, reflecting the nature of part-tasks for knowledge construction, meaning sharing and opinion sharing, and problem-solving. The results revealed greater accuracy for sharing activities and a higher level of collaborative performance. However, limitations remained in that the knowledge construction process and level of collaborative load were not considered at great depth (Fu et al., 2016; Yücel & Usluel, 2016). Therefore, this study explored the use of a tool for enhancing interaction processes by managing unnecessary loads in CSCL environments to facilitate meaningful discussion and knowledge construction. Method Participants We conducted an experimental study with 40 undergraduate students enrolled in an educational technology class at a university in Seoul, Korea. In accordance with the research ethics of the university in question, only students who were aware of and agreed to the purpose of the study participated in the experiment. Participation was voluntary and had no impact on their studies and grades. The learning environment for the three groups was designed with two independent variables provided before the problem-solving phase: type of sharing activities (meaning sharing activity (M) and opinion sharing activity (O)), and type of representation function to guide the sharing activities (word-based function (W) and sentence-based function (S)). The participants were randomly subdivided into three groups, and each group was provided with different types of VAT: • Group 1 was provided with M and O based on S (MSOS). • Group 2 was provided with M based on W and O based on S (MWOS). • The control group was provided with O based on S (OS). There were 14 participants in Group MSOS, 14 in Group MWOS, and 12 in Group OS. The average age of the students was 21.13. The distribution of gender was fairly even (52.5% males, 47.5% females), and participants were assigned to learning tasks in pairs. VAT for sequential knowledge construction Conventional discussion systems generally offer the learning material and an associated discussion in the same environment, even for complex tasks that involve part-tasks of different natures (Suthers & Hundhausen, 2003). Thus, these environments often impede students’ understanding of core concepts and hinder the sharing of specific content or opinions sequentially (Dillenbourg & Betrancourt, 2006; van Merriënboer & Sweller, 2005). To overcome the limitations of previous tools, this study focused on considering the nature of complex tasks and the specific functions of a representation tool to match this complexity (Eryilmaz et al., 2013). The representation tool we chose for this study is VAT, developed by Shin et al. (2018). It includes two types of sharing activities (meaning sharing and opinion sharing) before the problem-solving phase (see Table 1). The design of VAT takes into consideration the nature of complex tasks that involve congruent part-tasks (i.e., simple-to-complex tasks and core concept-to-specific content tasks) and specific functions Australasian Journal of Educational Technology, 2020, 36(4). 61 of the representation tool (i.e., differentiated representation functions for the two types of sharing activities) based on CLT and knowledge construction theory (Beers et al., 2005; Erilymaz et al., 2013). Table 1 Sub-tasks and functions of tools for each learning phase Learning phase Sub-task Function of tool Meaning sharing To define the meaning and explain the pros & cons of key words for understanding of core concepts Word-based annotation Opinion sharing To ask, submit of opinion and comment on specific content of learning material for understanding of learning contents Sentence-based annotation Problem-solving To discuss various opinions and derive solutions for completing the lesson-planning task Sentence-based annotation In the initial phase of learning, meaning sharing activities enable students to understand the core concept by clarifying and interpreting the meaning while identifying the pros and cons of the main terms. The VAT function for meaning sharing involves choosing a unit for a term and a fixed annotation box (e.g., a definition, pros and cons, and reference) (see Figure 1). If students choose a certain term from the learning material, they can organise their own knowledge and magnify their understanding by referencing others’ annotations. In the second phase of learning, an opinion sharing activity highlights all contributions concerning the details of the learning material. More extended interactions (e.g., assertion, support, and conflict) can enhance students’ expressions of opinion, flexibly reflecting previous contributions. In addition, the tool functions for opinion sharing activities enable learners to understand the specific learning content. If students choose a specified sentence or a paragraph from the learning materials, a more flexible annotation box for question and opinion would be provided (see Figure 2). Figure 1. Meaning sharing learning phase Figure 2. Opinion sharing learning phase In the final phase of learning, students perform a problem-solving activity in the form of lesson planning for an actual problem situation, adapting educational methodologies. Students can share and discuss based on first and second phase activities to facilitate descriptions of meaningful solutions through agreement and consensus building interactions. The tool function for problem-solving is the same as for the opinion sharing phase. Key word Definition Pros and cons Submit Cancel Meaning sharing phase Opinion sharing phase Annotation Type (Opinion) Title Message (Opinion) Click word unit Click sentence unit Key word box Australasian Journal of Educational Technology, 2020, 36(4). 62 Experiment design and procedure The experiment was conducted with three sub-tasks over four weeks in an online learning environment without face-to-face meetings. The participants first completed pre-tests that assessed prior knowledge, computer literacy, and collaborative tendencies. After conducting the pre-tests, learners were provided with the manual, which included the learning tasks and directions for using the tool. Working in pairs, the members of all groups performed sub-tasks using the same material related to educational methodology. Specifically, the sub-tasks consisted of core-concept understanding and sharing of learning material, specific-content sharing of learning material, and a lesson planning task. Students were asked to submit their lesson plans in pairs at the end of collaborative learning and then to measure collaborative load individually. Each learning phase was limited to one week (see Table 2). Table 2 Learning phases and learning methods based on visible-annotation types Visible- annotation type Sharing phase Comprehension of learning content task Problem-solving phase Lesson planning task 1st week 2nd week 3rd & 4th weeks MSOS Meaning sharing activity (1) Sub-task - To define the meaning of and explain core-concept of material (2) Function of tool - Sentence-based annotation - Linked annotation Opinion sharing activity (1) Sub-task - To ask, submit opinion and comment on specific content of material (2) Function of tool - Sentence-based annotation - Linked annotation Problem-solving activity (1) Sub-task - To negotiate various opinions and derive solutions for completing the lesson-planning task (2) Function of tool - Sentence-based annotation - Linked annotation MWOS Meaning sharing activity (1) Sub-task - To define the meaning and explain the pros & cons of core-concept of material (2) Function of tool - Word-based annotation - Linked annotation Opinion sharing activity (1) Sub-task - To ask, submit opinion and comment on specific content of material (2) Function of tool - Sentence-based annotation - Linked annotation OS Opinion sharing activity (1) Sub-task - To ask, submit opinion and comment on specific content of material (2) Function of tool - Sentence-based annotation - Linked annotation Pre-tests To assess participants’ prior knowledge, a multiple-choice test of five questions was administered (e.g., “Choosing the wrong explanation about the educational methodology”). To measure their level of prior computer literacy (e.g., “I have no difficulties in learning using the computer”) and prior collaborative preferences (e.g., “I prefer to share opinions with my friends”), a 5-point Likert scale was applied through a test of 10 questions. Knowledge construction activities This study combined quantitative content analysis and sequential analysis to examine students’ interactions during knowledge construction activities with an asynchronous discussion-based tool. The unit of analysis was the unit of meaning: sentences or phrases (Jeong, 2005). For content analysis, we adopted the coding scheme developed by Pena-Shaff and Nicholls (2004). To ensure inter-reliability of the analysis, three raters independently coded the messages. The reliability level was .80 (Cohen’s kappa). Australasian Journal of Educational Technology, 2020, 36(4). 63 Based on the quantitative content analysis results, we carried out a sequential analysis of students’ interactions using the discussion analysis tool (Jeong & Frazier, 2008). All of the segmented and coded units were arranged in chronological order. We then examined the result of transitional probabilities and diagrammed patterns to more closely examine interaction activities during the knowledge construction process. Figures 3, 4, and 5 depict transitional state diagrams that visually show the dynamic relationships between knowledge construction activities within a threaded discussion. The circles in each figure represent knowledge construction activities; the arrows between the circles indicate transitions. The thickness of an arrow is proportional to conversion probability, while the values specified in the figure show the conversion probability more specifically. Conversion probabilities of less than 0.30 were omitted from the figures to improve readability. Z-scores were also examined, taking into account not only the observed total number of responses to a particular message category but also the marginal total of each response type observed across all message types (Jeong, 2005). It was possible to systematically compare interaction probabilities across the three groups using z-scores to identify which interaction patterns occurred significantly more or less frequently than expected via the mean and standard deviation. If the probability of a pattern was low, this information could be used to diagnose interaction patterns, taking into account instructional context to more closely achieve the outcomes desired. Collaborative load Collaborative load was assessed using a 7-point Likert scale as developed in related cognitive studies (Janssen et al., 2010; Leppink et al., 2013), after submitting collaborative learning outcomes. All participants were asked to rate their perceived collaborative load on a scale ranging from extremely disagree (1) to extremely agree (7). The measurements consisted of seven items: three related to the instructional design of the tool (e.g., “There was no difficulty in understanding the collaboration process with team members”) and four related to collaborative work (e.g., “The way to derive conclusions with team members was satisfactory”). A reliability analysis revealed a Cronbach’s alpha value of 0.84. The ANCOVA method was conducted to control the results of three types of pre-tests as covariates (i.e., prior knowledge, prior computer literacy, and prior collaborative preferences). Collaborative learning outcomes Collaborative learning outcomes were assessed using a lesson-planning task related to instructional design principles in the problem-solving phase. A 3-point Likert scale was applied to measure the level of collaborative learning outcomes based on research by Dick, Carey, and Carey (2004) and Gagne, Wager, Golas, Keller, and Russell (2005). Two evaluators rated the level of collaborative learning outcomes using three scale values: poor (0), normal (0.5), and excellent (1). The evaluation criteria consisted of 13 ratings consisting of eight items related to frame (e.g., “The lesson plan is logical according to learning objectives and specific learning content) and five related to content (e.g., “The lesson plan offers appropriate learning activities for the learning strategy”). An inter-rater reliability analysis revealed a Cronbach’s alpha value of 0.91. The ANCOVA method was conducted to control of results of three types of pre-tests as covariates. Results Content analysis of knowledge construction activities The differences among the three groups in terms of quantitative results for discourse types in each learning phase were examined. The overall frequency of each type of discourse for Group MWOS was highest in all three phases. Specifically, in the meaning sharing phase, there was a statistically significant difference between Groups MSOS and MWOS (χ2 (6) = 69.18, p < 0.001). In particular, in Group MWOS, clarification and interpretation activities took place much more frequently than in Group MSOS. There was also a statistically significant difference among the three groups during the opinion sharing phase (χ2 (12) = 350.03, p < 0.001). However, for both Groups MSOS and MWOS, the order in which activities occurred was largely consistent as assertion, consensus building, and support. Nonetheless, MSOS differed by having more question activities, and Group MWOS had more conflict activities, though not by much when compared with the large proportion of other activities. Group OS, which was not provided with a meaning sharing phase, primarily engaged in assertion, clarification, and interpretation activities. The three groups also showed significant statistical differences in the problem- Australasian Journal of Educational Technology, 2020, 36(4). 64 solving phase (χ2 (10) = 141.84, p < 0.001), where Group MWOS completed more consensus building activities but fewer question activities than Group MSOS. Sequential analysis of knowledge construction activities Meaning sharing phase For Group MSOS, sequential analysis results for interactions in the meaning sharing phase showed that most activities, such as interpretation, questions, assertions, and support, converged towards clarification. However, the z-score result was only statistically significant for patterns from clarification to clarification (z = 2.78, p < .01). On the other hand, despite small cell frequencies, the results were significantly higher than expected for the pattern from conflict to question (z = 7.84, p < .01). In Group MWOS, all seven activities in the meaning sharing phase converged towards clarification. In addition, there were statistically significant results for the z-score from clarification to clarification (z = 2.78, p < .01), from interpretation to interpretation (z = 9.19, p < .01), from question to clarification (z = 1.81, p < .05), and from assertion to clarification (z = 2.19, p < .05). The interaction patterns of these two groups were similar in terms of clarification, but Group MWOS differed with significant z-scores for more various activities. In the case of Group MSOS, a pattern from conflict to question, which was expected to emerge during the opinion sharing phase, also appeared during the meaning sharing phase (see Figure 3). Figure 3. Transitional state diagram for the meaning sharing phases of Group MSOS (top) and MWOS (bottom) Opinion sharing phase For Group MSOS, sequential interaction results showed a pattern that converged towards assertion. In addition, the z-score of sequential analysis was statistically significant for patterns from assertion to assertion (z = 5.38, p < .01). Similar to the results for Group MSOS, the sequential pattern for Group MWOS also converged towards assertion activities. There were statistically significant results for the z- score from assertion to assertion (z = 3.14, p < .01). Although there were no significant differences in z- score results between Groups MSOS and MWOS, the proportion of patterns between discourse types differed in that more diverse types of discourse converged towards assertion activities for Group MWOS Australasian Journal of Educational Technology, 2020, 36(4). 65 than MSOS. In case of Group OS, there were significant results for patterns from clarification to clarification (z = 15.31, p < .01), from assertion to assertion (z = 14.77, p < .01), from conflict to question (z = 4.37, p < .01), and from support to support (z = 2.18, p <.05). Because Group OS was not provided with a meaning sharing phase, two types of discourse from convergent to divergent activities were shown at the same time in the opinion sharing phase (see Figure 4). Figure 4. Transitional state diagram for the opinion sharing phases of Group MSOS (top), Group MWOS (centre), and Group OS (bottom) Australasian Journal of Educational Technology, 2020, 36(4). 66 Problem-solving phase Finally, we examined how group discourse patterns in previous sharing activities affected the problem- solving process. In Group MSOS, the z-score result was significant from assertion to assertion (z = 10.27, p < .01) and from consensus building to consensus building (z = 9.88, p < .01). Patterns from conflict to conflict (z = 12.29, p < .01) were also noticeable but were not connected to other interaction activities. Group MWOS showed more inter-relation than Group MSOS, such as a relationship between assertion and consensus building (z = 4.93, p < .01). Although no significant results relating to conflict activities emerged, conflict was connected not only to itself but also to assertion activities. In Group OS, the z- score result was significant from assertion to assertion (z = 10.28, p < .01). Although conflict and support activities were shown to be significant in comparison with other groups, they were not connected to assertion or consensus building for conclusion derivations (see Figure 5). Figure 5. Transitional state diagram for the problem-solving phase of group MSOS (top left), Group MWOS (top right), and Group OS (bottom) Effects of types of learning tools on collaborative load Group MWOS had the highest score (M = 5.14, SD = 0.81) for optimising collaborative load, and Group MSOS had a lower score (M = 4.54, SD = 1.07) than Group OS (M = 4.80, SD = 0.91) (see Table 3). However, the result of ANCOVA analysis showed that there was no significant difference in the collaborative load among groups [F (2, 34) = 1.09, p = 0.35, ηp2 = 0.00]. Table 3 Effects of types of tool on collaborative load measures Types of tool Collaborative load M SD AM MSOS 4.54 1.07 4.60 MWOS 5.14 0.81 5.14 OS 4.80 0.91 4.74 N = 40. M = Mean. SD = Standard deviation. AM = Adjusted mean. Australasian Journal of Educational Technology, 2020, 36(4). 67 Effects of types of learning tools on collaborative learning outcomes The results of ANCOVA for constructed knowledge found significant differences between conditions [F (2, 34) = 6.07, p = 0.01, ηp2 = 0.26] (see Table 4), with the highest collaborative learning outcome achieved by Group MWOS (M = 10.86, SD = 1,49) (see Table 5). The results suggest that the meaning sharing phase for the learning content could affect the level of collaborative learning outcomes. Table 4 Covariance analysis of collaborative learning outcome by visible-annotation tool type Source SS df MS F p Model 88.42 5 17.69 2.87 0.03 Computer usability 1.69 1 1.69 0.28 0.60 Prior knowledge 1.13 1 1.13 0.18 0.67 Collaborative tendency 0.02 1 0.02 0.00 0.96 Tool type 74.65 2 37.33 6.07 0.01 Error 209.18 34 6.15 Total 297.60 39 N = 40, SS = Sum of square, MS = Mean square, **p < .05 Table 5 Effects of types of tool on collaborative learning outcomes measures Types of tool M SD AM MSOS 8.04 2.72 8.07 MWOS 10.86 1.49 10.83 OS 7.50 2.66 7.41 N = 40. M = Mean. SD = Standard deviation. AM = Adjusted mean. Discussion Knowledge construction activities The findings of this study indicate that the two groups provided with a meaning sharing phase, MSOS and MWOS, actively demonstrated clarification and interpretation activities that helped facilitate accurate sharing of core concepts to complete a complex task. In particular, Group MWOS, with a word-based function, showed much more frequent clarification and interpretation activities than Group MSOS, with a word-based function. Based on the results, providing a word-based function in the meaning sharing phase is more effective for facilitating interactions to identify key concepts. In the opinion sharing phase, Groups MSOS and MWOS engaged in assertion, consensus building and support activities, while Group OS completed assertion, clarification and interpretation activities. This suggests that Group OS was not provided with a meaning sharing phase, which might lead learners engaged in identifying the meaning of learning concepts into sharing different ideas at the same time. More dynamic sharing of opinions to understand learning content occurred in Groups MSOS and MWOS, receiving meaning sharing activities. In the problem-solving phase, enough assertion and consensus building took place to induce meaningful agreement for solving a task. Given that more consensus building activities and fewer question activities occurred in Group MWOS than in Group MSOS, it seems that Group MWOS more effectively facilitated active interactions for building common ground and integrating diverse opinions, which enabled learners to reach consensus efficiently in the problem-solving phase. Overall, Group MWOS was found to most effectively generate active interactions for knowledge sharing and problem-solving in a CSCL environment. The results indicate that a strategy to promote clarification and interpretation activities is necessary to encourage learners to identify key words during the meaning sharing phase. In addition, a word-based function in the meaning sharing phase can promote more focused sharing of core concepts, which leads to more frequent interpretation and sharing of learning concepts. This can help promote dynamic ideas in the opinion sharing phase. This result is consistent with the findings of previous studies, which suggested that it is necessary to provide an instructional strategy that matches the learning phase Australasian Journal of Educational Technology, 2020, 36(4). 68 and the characteristics of learning content to enable successful collaborative learning (Jung, Shin, & Zumbach, 2019; Slof et al., 2010). Concerning the pattern of movement between communication activities, in the meaning sharing phase, conversations tended to concentrate towards clarification for both Groups MSOS and MWOS. In particular, Group MWOS showed effective concept learning with significant patterns from clarification to clarification, from interpretation to interpretation, from question to clarification and from assertion to clarification. During the opinion sharing process, Groups MSOS and MWOS showed similar patterns of interaction concentrating towards assertion, while Group OS showed various types of discourse converging towards assertion, questions, interpretation, and clarification. Because Group OS was not provided with a meaning sharing phase, it seemed learners responded to activities for identifying the meaning of concepts and discussing different ideas to understand the learning content at the same time. In the problem-solving phase, there was little interaction between consensus building and assertion for Group MSOS compared to Group MWOS. In the case of Group MWOS, patterns from consensus building to assertion and from assertion to consensus building were significant. In-depth consensus building was conducted in order to form common ground by organising the content of preceding discussions, explaining previous opinions in greater detail, asking for agreement, and expressing degrees of agreement in detail. Based on the results, Group MWOS was found to be most effective at handling meaningful interactions, showing diverse discourse patterns for each learning phase in the collaborative knowledge-construction process. How individuals share knowledge and the accuracy with which this knowledge sharing is conducted is essential and has an impact on the collaborative knowledge construction process (Bromme, 2000). These findings are in line with those of previous studies that revealed the effects of appropriate learning strategies considering collaborative knowledge construction in a CSCL environment (Jung et al., 2019; Shin et al., 2018). Collaborative load Collaborative load was not statistically significant but was higher for Groups MSOS, OS, and MWOS, in descending order. This result indicates that Group MSOS, which engaged in meaning sharing and opinion sharing activities with a sentence-based function, was less effective in reducing collaborative load than Group MWOS, which engaged in opinion sharing with a sentence-based function, after completing meaning sharing with a word-based function. Although Group MWOS showed enhanced interactions for constructing collaborative knowledge, it was difficult to verify the significance of the tools employed for lowering collaborative load compared to other groups. Collaborative outcomes Collaborative outcomes were higher for Groups MWOS, MSOS, and OS, in descending order. Considering that Group MWOS engaged in more interactions than Groups MSOS and OS in all three learning phases, the knowledge construction activities and interactions that took place for Group MWOS are assumed to have been meaningful, leading to effective collaborative learning outcomes. CSCL is effective when communication activities that are valuable for learning occur (Dillenbourg & Hong, 2008). However, it is not easy for learners with diverse perspectives to share their ideas with each other and build shared knowledge successfully in a CSCL environment. In particular, meaningful interactions are unlikely to be achieved if learners fail to accurately share knowledge and ideas, while meaningless interactions may impose unnecessary effort on learners, interfere with communicative activities, and lead to collaborative learning failing (Barron, 2003; Erilymaz et al., 2013). Applying different supportive tools or functions for each learning phase is very important for the whole process of complex learning (Jung et al., 2019; Slof et al., 2010), so an effective instructional strategy should be introduced to support learners in each specific phase for inducing meaningful interaction patterns in proper sequence from simple to complex. This study provides empirical evidence that a strategy implementing a word-based function in the meaning sharing phase can effectively facilitate concept learning, resulting in meaningful communication activities in other learning phases and contributing to successful collaborative outcomes. Australasian Journal of Educational Technology, 2020, 36(4). 69 Conclusion and further research Representation tools are effective in reducing unnecessary collaboration load and leading to meaningful interactions (Kolloffel et al., 2011). However, several studies have pointed out that it is not easy for learners to communicate with each other to build accurate shared knowledge using existing representation tools (Barron, 2003; Eryilmaz et al., 2009; Slof et al., 2012; Suthers & Hundhausen, 2003). All the same, most studies have not focused on the relationship between the accuracy of shared knowledge and the level of collaborative performance (Shin et al., 2018). To overcome the limitations of previous studies, this study investigated collaborative processes and collaborative outcomes. Based on the results, it can be concluded that MWOS contributes to the occurrence of effective interactions, collaborative load, and collaborative learning outcomes. In particular, this study analysed the knowledge construction process more systematically than prior research based on a theoretical frame of prior research (e.g., Beers et al., 2005; Eryilmaz et al., 2013; Shin et al., 2018). The resulting implications are that the representation tool VAT enhances solution effectiveness when consideration is given to the nature of part-tasks, as shown through sequential discourse patterns. In further planned research, it will be necessary to conduct an empirical study with larger sample sizes to generalise the results derived from this study. It will be necessary to verify the effectiveness of using VAT for a variety of topics and learning environments. In addition, given the results of the discourse pattern, which did not address divergent interactions after the meaning sharing phase, more sophisticated tool function design is needed to derive more meaningful interactions and creative solutions. Acknowledgements This work was supported by NRF (National Research Foundation of Korea) Grant funded by the Korean Government (NRF-2016-Fostering Core Leaders of the Future Basic Science Program/Global Ph.D. Fellowship Program). References Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359. https://doi.org/10.1207/S15327809JLS1203_1 Beers, P. J., Boshuizen, H. P. E., Kirschner, P. A., & Gijselaers, W. H. (2005). Computer support for knowledge construction in collaborative learning environments. Computers in Human Behavior, 21(4), 623–643. https://doi.org/10.1016/j.chb.2004.10.036 Bell, P. (1997). Using argument representations to make thinking visible. In R. Hall, N. Miyake, & N. Enyedy (Eds.), Proceedings of CSCL '97: The Second International Conference on Computer Support for Collaborative Learning (pp. 10–19). Toronto, Canada: University of Toronto Press. Bernard, R. M., & Lundgren-Cayrol, K. (2001). Computer conferencing: An environment for collaborative project-based learning in distance education. Educational Research and Evaluation, 7(2- 3), 241–261. https://doi.org/10.1076/edre.7.2.241.3866 Bromme, R. (2000). Beyond one’s own perspective: The psychology of cognitive interdisciplinary. In P. Weingart & N. Stehr (Eds.), Practicing interdisciplinary (pp. 115–133). Toronto, Canada: Toronto University. Bruggen, J. M., Kirschner, P. A., & Jochems, W. (2002). External representation of argumentation in CSCL and the management of cognitive load. Learning and Instruction, 12(1), 121–138. https://doi.org/10.1016/S0959-4752(01)00019-6 Cacciamani, S., Cesareni, D., Martini, F., Ferrini, T., & Fujita, N. (2012). Influence of participation, facilitator styles, and metacognitive reflection on knowledge building in online university courses. Computers & Education, 58(3), 874–884. https://doi.org/10.1016/j.compedu.2011.10.019 Dick, W., Carey, L., & Carey, J. (2004). Systematic design of instruction. Upper Saddle River, NJ: Pearson. Dillenbourg, P., & Betrancourt, M. (2006). Collaboration load. In J. Elen & R. E. Clark (Eds.), Handling complexity in learning environments: Research and theory (pp. 142–163). Amsterdam, The Netherlands: Elsevier. Dillenbourg, P., & Hong, F. (2008). The mechanics of CSCL macro scripts. International Journal of Computer-Supported Collaborative Learning, 3(1), 5–23. https://doi.org/10.1007/s11412-007-9033- 1 https://doi.org/10.1207/S15327809JLS1203_1 https://doi.org/10.1016/j.chb.2004.10.036 https://doi.org/10.1076/edre.7.2.241.3866 https://doi.org/10.1016/S0959-4752(01)00019-6 https://doi.org/10.1016/j.compedu.2011.10.019 https://doi.org/10.1007/s11412-007-9033-1 https://doi.org/10.1007/s11412-007-9033-1 Australasian Journal of Educational Technology, 2020, 36(4). 70 Engelmann, T., & Hesse, F. W. (2010). How digital concept maps about the collaborators’ knowledge and information influence computer-supported collaborative problem solving. International Journal of Computer-Supported Collaborative Learning, 5(3), 299–319. https://doi.org/10.1007/s11412-010- 9089-1 Eryilmaz, E., Alrushiedat, N., Kasemvilas, S., Mary, J., & Pol, J. (2009). The effect of anchoring online discussion on collaboration and cognitive load. In Proceedings of the 15th Americas Conference on Information Systems (art. 576). St. Petersburg, FL: Association for Information Systems. Retrieved from http://aisel.aisnet.org/amcis2009/576 Eryilmaz, E., van der Pol, J., Ryan, T., Clark, P. M., & Mary, J. (2013). Enhancing student knowledge acquisition from online learning conversations. International Journal of Computer-Supported Collaborative Learning, 8(1), 113–144. https://doi.org/10.1007/s11412-012-9163-y Fransen, J., Weinberger, A., & Kirschner, P. A. (2013). Team effectiveness and team development in CSCL. Educational Psychologist, 48(1), 9–24. https://doi.org/10.1080/00461520.2012.747947 Fu, E. L., van Aalst, J., & Chan, C. K. (2016). Toward a classification of discourse patterns in asynchronous online discussions. International Journal of Computer-Supported Collaborative Learning, 11(4), 441–478. https://doi.org/10.1007/s11412-016-9245-3 Fuerstenau, B., Ryssel, J., & Kunath, J. (2010). Concept mapping versus summary writing as instructional devices for understanding complex business problems. In S. Goldman, J. Pellegrino, K. Gomez, L. Lyons, & J. Radinsky (Eds.), Learning in the Disciplines: Proceedings of the 9th International Conference of the Learning Sciences (vol. 2, pp. 14–16), Chicago, IL: International Society of the Learning Sciences. Retrieved from https://dl.acm.org/citation.cfm?id=1854509 Gagne, R. M., Wager, W. W., Golas, K. C., Keller, J. M., & Russell, J. D. (2005). Principles of instructional design. Performance Improvement, 44(2), 44–46. https://doi.org/10.1002/pfi.4140440211 Hull, D. M., & Saxon, T. F. (2009). Negotiation of meaning and co-construction of knowledge: An experimental analysis of asynchronous online instruction. Computers & Education, 52(3), 624–639. https://doi.org/10.1016/j.compedu.2008.11.005 Janssen, J., Kirschner, F., Erkens, G., Kirschner, P. A., & Paas, F. (2010). Making the black box of collaborative learning transparent: Combining process-oriented and cognitive load approaches. Educational Psychology Review, 22(2), 139–154. https://doi.org/10.1007/s10648-010-9131-x Jeong, A. (2005). A guide to analyzing message–response sequences and group interaction patterns in computer‐mediated communication. Distance Education, 26(3), 367–383. https://doi.org/10.1080/01587910500291470 Jeong, A., & Frazier, S. (2008). How day of posting affects level of critical discourse in asynchronous discussions and computer‐supported collaborative argumentation. British Journal of Educational Technology, 39(5), 875–887. https://doi.org/10.1111/j.1467-8535.2007.00789.x Jung, J., Shin, Y., & Zumbach, J. (2019). The effects of pre-training types on cognitive load, collaborative knowledge construction and deep learning in a computer-supported collaborative learning environment. Interactive Learning Environments, 1–13. https://doi.org/10.1080/10494820.2019.1619592 Kirschner, F., Paas, F., & Kirschner, P. A. (2009). A cognitive load approach to collaborative learning: United brains for complex tasks. Educational Psychology Review, 21(1), 31–42. https://doi.org/10.1007/s10648-008-9095-2 Kolloffel, B., Eysing, T. H., de Jong, T. (2011). Comparing the effects of representational tools in collaborative and individual inquiry learning. International Journal of Computer-Supported Collaborative Learning, 6(2), 223–251. https://doi.org/10.1007/s11412-011-9110-3 Leppink, J., Paas, F., Van der Vleuten, C. P., Van Gog, T., & Van Merriënboer, J. J. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45(4), 1058–1072. https://doi.org/10.3758/s13428-013-0334-1 Ludvigsen, S. (2016). CSCL towards the future: The second decade of ijCSCL. International Journal of Computer-Supported Collaborative Learning, 11(1), 1–7.https://doi.org/10.1007/s11412-016-9230-x Nelson, B. C., & Erlandson, B. E. (2008). Managing cognitive load in educational multi-user virtual environments: Reflection on design practice. Educational Technology Research and Development, 56(5-6), 619–641. https://doi.org/10.1007/s11423-007-9082-1 Pena-Shaff, J. B., & Nicholls, C. (2004). Analyzing student interactions and meaning construction in computer bulletin board discussions. Computers & Education, 42(3), 243–265. https://doi.org/10.1016/j.compedu.2003.08.003 https://doi.org/10.1007/s11412-010-9089-1 https://doi.org/10.1007/s11412-010-9089-1 http://aisel.aisnet.org/amcis2009/576 https://doi.org/10.1007/s11412-012-9163-y https://doi.org/10.1080/00461520.2012.747947 https://doi.org/10.1007/s11412-016-9245-3 https://dl.acm.org/citation.cfm?id=1854509 https://doi.org/10.1002/pfi.4140440211 https://doi.org/10.1016/j.compedu.2008.11.005 https://doi.org/10.1007/s10648-010-9131-x https://doi.org/10.1080/01587910500291470 https://doi.org/10.1111/j.1467-8535.2007.00789.x https://doi.org/10.1080/10494820.2019.1619592 https://doi.org/10.1007/s10648-008-9095-2 https://doi.org/10.1007/s11412-011-9110-3 https://doi.org/10.3758/s13428-013-0334-1 https://doi.org/10.1007/s11412-016-9230-x https://doi.org/10.1007/s11423-007-9082-1 https://doi.org/10.1016/j.compedu.2003.08.003 Australasian Journal of Educational Technology, 2020, 36(4). 71 Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In C. O’Malley (Ed.), Computer supported collaborative learning (pp. 69–97). Berlin, Germany: Springer. https://doi.org/10.1007/978-3-642-85098-1_5 Schellens, T., & Valcke, M. (2006). Fostering knowledge construction in university students through asynchronous discussion groups. Computers & Education, 46(4), 349–370. https://doi.org/10.1016/j.compedu.2004.07.010 Seufert, T., & Brünken, R. (2006). Cognitive load and the format of instructional aids for coherence formation. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 20(3), 321–331. https://doi.org/10.1002/acp.1248 Shin, Y., Kim, D., & Jung, J. (2018). The effects of representation tool (Visible-annotation) types to support knowledge building in computer-supported collaborative learning. Journal of Educational Technology & Society, 21(2), 98–110. Retrieved from http://www.jstor.org/stable/26388383 Slof, B., Erkens, G., Kirschner, P. A., Janssen, J., & Jaspers, J. G. M. (2012). Successfully carrying out complex learning-tasks through guiding teams’ qualitative and quantitative reasoning. Instructional Science, 40(3), 623–643. https://doi.org/10.1007/s11251-011-9185-2 Slof, B., Erkens, G., Kirschner, P. A., Jaspers, J. G., & Janssen, J. (2010). Guiding students’ online complex learning-task behavior through representational scripting. Computers in Human Behavior, 26(5), 927–939. https://doi.org/10.1016/j.chb.2010.02.007 Stahl, G. (2015). A decade of CSCL. International Journal of Computer-Supported Collaborative Learning, 10(4), 337–344. https://doi.org/10.1007/s11412-015-9222-2 Suthers, D. D., & Hundhausen, C. D. (2003). An experimental study of the effects of representational guidance on collaborative learning processes. The Journal of the Learning Sciences, 12(2), 183–218. https://doi.org/10.1207/S15327809JLS1202_2 Suthers, D. D., Vatrapu, R., Medina, R., Joseph, S., & Dwyer, N. (2008). Beyond threaded discussion: Representational guidance in asynchronous collaborative learning environments. Computers & Education, 50(4), 1103–1127. https://doi.org/10.1016/j.compedu.2006.10.007 van Merriënboer, J. J., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177. https://doi.org/10.1007/s10648-005-3951-0 Wang, Z., Anderson, T., Chen, L., & Barbera, E. (2017). Interaction pattern analysis in cMOOCs based on the connectivist interaction and engagement framework. British Journal of Educational Technology, 48(2), 683–699. https://doi.org/10.1111/bjet.12433 Yücel, Ü. A., & Usluel, Y. K. (2016). Knowledge building and the quantity, content and quality of the interaction and participation of students in an online collaborative learning environment. Computers & Education, 97, 31–48. https://doi.org/10.1016/j.compedu.2016.02.015 Corresponding author: Jaewon Jung, jjungj5@gmail.com Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY- NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Please cite as: Shin, Y., & Jung, J. (2020). The effects of a visible-annotation tool for sequential knowledge construction on discourse patterns and collaborative outcomes. Australasian Journal of Educational Technology, 36(4), 57–71. https://doi.org/10.14742/ajet.4875 https://doi.org/10.1007/978-3-642-85098-1_5 https://doi.org/10.1016/j.compedu.2004.07.010 https://doi.org/10.1002/acp.1248 http://www.jstor.org/stable/26388383 https://doi.org/10.1007/s11251-011-9185-2 https://doi.org/10.1016/j.chb.2010.02.007 https://doi.org/10.1007/s11412-015-9222-2 https://doi.org/10.1207/S15327809JLS1202_2 https://doi.org/10.1016/j.compedu.2006.10.007 https://doi.org/10.1007/s10648-005-3951-0 https://doi.org/10.1111/bjet.12433 https://doi.org/10.1111/bjet.12433 https://doi.org/10.1016/j.compedu.2016.02.015 mailto:jjungj5@gmail.com https://creativecommons.org/licenses/by-nc-nd/4.0/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.4875 Introduction Literature review The importance of exploring the knowledge construction process in CSCL Collaborative load in CSCL The effect of a representation tool on collaborative learning Method Participants VAT for sequential knowledge construction Experiment design and procedure Pre-tests Knowledge construction activities Collaborative load Collaborative learning outcomes Results Content analysis of knowledge construction activities Sequential analysis of knowledge construction activities Meaning sharing phase Opinion sharing phase Problem-solving phase Effects of types of learning tools on collaborative load Effects of types of learning tools on collaborative learning outcomes Discussion Knowledge construction activities Collaborative load Collaborative outcomes Conclusion and further research Acknowledgements References