Australasian Journal of Educational Technology, 2021, 37(2). 81 Sentiment evolution with interaction levels in blended learning environments: Using learning analytics and epistemic network analysis Changqin Huang, Zhongmei Han, Ming Li, Xizhe Wang, Wenzhu Zhao Zhejiang Normal University, Jinhua, China Sentiment evolution is a key component of interactions in blended learning. Although interactions have attracted considerable attention in online learning contexts, there is scant research on examining sentiment evolution over different interactions in blended learning environments. Thus, in this study, sentiment evolution at different interaction levels was investigated from the longitudinal data of five learning stages of 38 postgraduate students in a blended learning course. Specifically, text mining techniques were employed to mine the sentiments in different interactions, and then epistemic network analysis (ENA) was used to uncover sentiment changes in the five learning stages of blended learning. The findings suggested that negative sentiments were moderately associated with several other sentiments such as joking, confused, and neutral sentiments in blended learning contexts. Particularly in relation to deep interactions, student sentiments might change from negative to insightful ones. In contrast, the sentiment network built from social-emotion interactions shows stronger connections in joking-positive and joking-negative sentiments than the other two interaction levels. Most notably, the changes of co-occurrence sentiment reveal the three periods in a blended learning process, namely initial, collision and sublimation, and stable periods. The results in this study revealed that students’ sentiments evolved from positive to confused/negative to insightful. Implications for practice or policy: • Learning analytics can be used to identify the sentiments and interactions from discussions. • Instructors should guide students to experience slightly negative and confused sentiments for deep interactions at the beginning of blended learning. • Social-emotion interactions can alleviate the influence caused by confused sentiments when completing learning activities. • Deep interactions can play an important role in improving problem-solving abilities, and when problems are settled, sentiments shift from negative/confused to positive/insightful sentiments. Keywords: sentiment evolution, interaction levels, learning analysis, epistemic network analysis Introduction Blended learning, which is the integration of online activities and a classroom paradigm, offers students the advantage of flexibility in time and space (Owston et al., 2019). Fueled by new technology trends, there are growing demands to leverage advanced techniques to support blended learning in smart learning environments, for example, online discussions, resource sharing and recommendations (Rasheed et al., 2020). Such smart learning environments, perceived as a support for technology-enhanced blended learning, not only enable students to interact with learning systems/their peers anywhere and at any time, also provide digital resources, learning guidance, hints, and supportive tools (Cocquyt et al., 2019). Discussion is regarded as a primary, and widely used, avenue for knowledge construction in blended learning contexts (Han & Ellis, 2019). As Law et al. (2019) pointed out, students can enhance their diverse abilities (e.g., problem-solving, social negotiation, and sentiment engagements) by interactive discussion with each other in blended learning contexts. Hence, interactions among learners can harness the benefits of discussions to deliver high-quality experiences of blended learning. Interactions, a key component of blended learning, evaluate the learning level of involvements of those performing learning tasks (Liu & Shi, 2018). A large number of studies have been conducted on interactions Australasian Journal of Educational Technology, 2021, 37(2). 82 to better understand the learning process in blended learning (Boelens et al., 2017; Muñoz-Cristóbal et al., 2018; Shu & Gu, 2018). Boelens et al. (2017) focused on learner interactions to explain learning changes in blended learning. Moreover, some researchers considered social interactions to examine the sequences of multiple interactions during the process of knowledge construction (Huang et al, 2019). In the interaction process, students’ sentiments are dynamic and unfold over time in blended learning contexts (Pérez-López et al., 2020). Therefore, the interaction process cannot be fully understood without examining sentiments embodied in interactions (Garcia et al., 2016). However, only a few attempts have been made to examine deeply how sentiment evolution contributes to the level of interactions during the process of constructing knowledge. As such, a core issue is to examine how sentiments evolve with different levels of interactions in blended learning contexts. More recently, considerable emphasis has been placed on adopting diverse analysis methods to identify sentiments and interactions in blended learning environments. To recognise the sentiments and interactions in discussion messages, learning analysis has been proven to be more effective than traditional self-reported and manually coded methods. Particularly in relation to text analysis, learning analytics has been integrated into the mainstream of sentiment analysis, such as natural language processing (Chatterjee et al., 2019). Apart from classification, the dynamic analysis of sentiments and interactions has drawn increasing attention. Network analysis approaches (e.g., social network analysis, epistemic network analysis [ENA]) are widely employed to demonstrate the connections between aspects of students’ knowledge construction (Bressler et al., 2019). In addition to modelling the connections between sentiments and interactions, ENA can quantify and produce a weighted graph of co-occurrences with visualisations (Rolim et al., 2019). Thus, ENA enables researchers to understand not only how qualitative codes are connected but also how those connections vary with different levels of interactions (Shaffer et al., 2016). However, the rather limited body of research involves combinations of text mining and ENA in the longitudinal exploration of sentiment evolution with different levels of interactions. Therefore, this study used text mining and ENA to investigate sentiment evolution with varying levels of interactions throughout different learning stages in blended learning contexts. Literature review Blended learning Over the past decade, significant technological advancements have resulted in the increasing adoption of online and blended modes of learning (Asarta & Schmidt, 2020). Blended learning has been widely deployed, being a combination of e-learning and the traditional classroom paradigm, especially in the application of smart learning environments (Han & Ellis, 2019). As suggested by Asarta and Schmidt (2020), a student’s experience in a classroom relies heavily on the online materials that might improve learning performance. Hence, blended learning is a new pedagogical practice that mixes various activities to create constructive learning. The online support for blended learning has become very suitable to encourage students to solve problems (Rasheed et al., 2020). The vital responsibility of online learning resource flipped classrooms as a typical blended model is to provide diverse learning materials for knowledge construction (Simmons et al., 2020). Wang et al. (2020) examined the structure of students’ tendency characteristics in small private open course with the support of peer reviews in a blended learning course. Despite the impetus to build blended learning, many researchers examine learning performance from the perspective of learning achievements (e.g., score) (Asarta & Schmidt, 2020). Celestial-Valderama et al. (2021) mined students’ process feedback to unravel the improvements in a blended learning course. In addition, integrating blended classroom lectures with online discussions is often severely constrained by learning difficulties in the high-quality interactions among students and sentiment lacks (Rasheed et al., 2020). Therefore, it is necessary to investigate the learning process (e.g., interaction, sentiments) to facilitate knowledge constructions in blended learning contexts. Interactions and sentiments Interactions Several concerns have been raised in relation to analysing interactions in educational settings. These interactions occur between the students, with the course content, and with the course instructors Australasian Journal of Educational Technology, 2021, 37(2). 83 (Purarjomandlangrudi & Chen, 2020). In prior studies, various interaction analysis frameworks were employed to better understand the interaction process. Interactions in blended learning settings occur not just between the instructor and student, but also between the learning content and the platforms (Boelens et al., 2017). The study by Kurucay and Inan (2017) found that there were three types of interactions among instructors, students and learning content (in the case of delivering the learning content), namely instructors-students, instructors-learning content, and students-learning content in online learning contexts. Apart from focusing on the relationship between the interactions, Yang et al. (2018) employed an interaction analysis model in terms of knowledge construction. The interaction analysis model consisted of five categories: (1) sharing/comparing information; (2) discovery and exploration of dissonance; (3) negotiation; (4) test and modification; and (5) agreement statement. To understand the cognitive process, these five categories of interactions were further classified from surface to deep interactions according to the framework proposed by Wang et al. (2014). Sentiments A large body of work has reported that there are links between different levels of interactions and sentiments. Sentiments reflect students’ learning attitude during their knowledge construction (Ortigosa et al., 2014). Rubenking (2019) examined how sentiments influenced the first type of interaction analysis model (sharing intentions) in online environments. In blended learning environments, there has been much interest in examining the effect of learning sentiments (Marchand & Gutierrez, 2012). Compared with online and face-to-face teaching, the results from a study conducted by Han et al. (2020) revealed that to a great extent, sentiments could be expressive of the learning judgment of the learning process. Learning sentiments are embedded in interactive discussions in blended learning. In particular, much attention has been paid to identify the sentiments that occur in discussions. Thus, an important research direction is to explore the variety of sentiments experienced in blended learning discussions. Before identifying the sentiments to examine the interaction process, it is relevant to give a taxonomy of sentiments. Generally, learning sentiments consist of three types, that is, positive (e.g., happy, enjoyment), negative (e.g., frustration, anxiety, boredom), and neutral (Pekrun, 2006). Liu et al. (2019) discovered positive, negative, and confused sentiments from the textual data produced in online course forums. Similar to interaction text data, six-dimensional sentiments were found by the study of Zheng and Huang (2016), namely: positive, negative, neutral, insightful, confused, and joking. These six types of sentiments are tightly woven together throughout blended learning to assist the students to construct knowledge. By following the improved classification of the six-dimensional sentiments (Zheng & Huang, 2016), this study attempted to provide further insights into learning sentiments in blended learning contexts. The relationship between interactions and sentiments Recently, much attention has been paid to the close links between sentiments and interactions. Sentiments are often regarded as a vital indicator of knowledge construction in establishing effective interactions (Wang et al., 2017). For example, as explicated by Huang et al. (2019), different interaction levels might evoke diverse sentiments. Their results indicated that surface and deep interactions were associated with confusion sentiments, such as disagreement and negotiation. Additionally, dynamic sentiments experienced in the interaction process have a significant influence on learning effectiveness (Kort et al., 2001). Huang et al. (2019) proposed a 4-phase model for describing the interplay process between sentiments and interactions. Their model could be used to shed light on the investigation of changes in sentiment evolution in blended learning environments, which divided the learning process into five stages, including generation, collision and integration, refinement and stability of interaction with learning sentiments. Thus, recent work in dynamic sentiments has suggested that shifts in sentiment and interactions could provide a deep understanding of blended learning (Manwaring et al., 2017). However, little research has been devoted to understanding how sentiments change with the interactions in the blended learning process and thus sentiments evolution is not clearly understood. In this regard, it is worth uncovering how students’ sentiments vary over the interaction levels in a blended learning course. Learning analytics and epistemic network analysis Sentiment and interaction analysis are important, with applications ranging from the automated analysis of discussion messages to understanding sentiment evolution over different interaction levels. Several scholars have employed diverse methods to identify the sentiments and interactions. For instance, Yang et al. (2018) employed content analysis to code the online interactions to investigate online interaction patterns. While Australasian Journal of Educational Technology, 2021, 37(2). 84 significant efforts have been made to manually code sentiments and interactions (Huang et al., 2019), the analysis carried out thus far has primarily been very time consuming in the context of blended learning. For the most part, it is essential to automatically recognise sentiments and interactions. Human validation experiments indicate that machine derived sentiments are correlated closely with the sentiments derived from human coded data (Hew et al., 2020). For this to happen in a context such as discussion messages, we need to first automate the classifications of blended discussions to identify categories of sentiment and interaction dimensions. There are promising directions in the automated analysis of discussions that can be helpful (Paulus & Wise, 2019). Tang et al. (2016) used long short-term memory (LSTM) to recognise interactions in online learning. Using feature extraction, LSTM has been increasingly leveraged in education to provide rich insights from extensive discussions. In education research, LSTM has been applied to many problems such as essay grading (Liang et al., 2018) and massive open online course interaction modelling (Wei et al., 2017). As pointed out by Cabrera et al. (2015), qualitative analysis is a way of indirectly observing the interactive and cognitive process. In terms of understanding of how students’ sentiments evolve and vary with different interaction levels in a network, few insights were offered by the above analysis methods on sentiments and interactions. Epistemic network analysis (ENA) can model variables (e.g., sentiments, interactions) and their relationships in a network by the quantitative analysis of discussions (Bressler et al., 2019; Csanadi et al., 2018; Misiejuk et al., 2021). In particular, ENA can be used to investigate the progressive relationships between these variables (Hod et al., 2020). The three core concepts of ENA are: codes, units, and sections (Csanadi et al., 2018). A set of conceptual elements are denoted as ENA codes (e.g., sentiments and interactions). ENA objects are used to represent the analysis units, such as learning phases (e.g., stages of blended learning) or group divisions (e.g., interaction levels). The occurrence scope of codes represents the sections. In general, ENA is used to investigate the connections reflecting the co-occurrence of codes (e.g., sentiments extracted from students’ interactions) where a coding scheme is applied to analyse online discussion messages (Bressler et al., 2019; Wu et al., 2020). With this analysis, the statistical differences between groups of interaction levels can be found. ENA has the advantage of being able to explain the main characteristics that led to these differences. Therefore, this study adopted ENA to analyse sentiment evolution with different interaction levels during blended learning. Research questions Although highly crucial for an understanding of sentiments and interactions when constructing knowledge in blended learning, there is a dearth of research that investigates how interactions are related to the sentiments that shape the social network structure. In particular, the emerging temporal learning analytics face theoretical and technical challenges in analysing time-series interaction data and making educational sense of the results. However, ENA can capture the dynamic network of sentiments. There is a lacuna in the literature investigating sentiment evolution with different levels of interactions using text mining and ENA. In this regard, this study was guided by the following research questions: 1. What are the characteristics of sentiments at different levels of interactions while engaging in knowledge construction activities in blended learning? 2. In what ways are sentiments connected and changing at different levels of interactions, as illustrated by an ENA model derived from online discussions in blended learning? 3. How do sentiments evolve at the different stages of blended learning? Method Participants A total of 38 postgraduate students (31 females and 7 males) studied a blended course, Fundamental Educational Technology Theory Course, in a university in China. Their ages ranged from 22 to 31 years (mean = 24.40), with their major in education technology. All participants already had rich online learning experiences before this experiment. Australasian Journal of Educational Technology, 2021, 37(2). 85 Blended learning contexts The main learning objective of the blended course was to understand and apply technology in education. Before studying the blended course, all the participants received training on how to use the learning platform, “Liru Cloud Classroom”. Delivered in both mobile-based and web-based versions (http://moodle.scnu.edu.cn), the platform provides students with learning materials, interactions by online discussions, information/notices/homework and assignment submission to promote problem-solving during blended learning. It enables flexible learning from any place within a specified area and the display style of the subject content can be adapted to the requirements of different mobile devices. Under this learning platform, students can discuss subject issues without the constraints of time and space. For example, comments and replies during their knowledge construction via the asynchronous discussion forum in the platform can occur anytime. Students were requested to complete seven learning activities online and offline over the whole blended course where offline learning (face-to-face) was interspersed among the online tasks, as shown in Figure 1. Figure 1. The blended learning process detailing online and face-to-face modes The instructor and the researcher co-designed the online and face-to-face learning activities illustrated in Figure 1 to enable the participants to have a thorough understanding of educational technology. The participants engaged in these learning tasks, step by step for 15 weeks. The learning activities were categorised into online discussion, face-to-face learning, debate, and task debriefing in class. In the classroom learning setting, each student could freely express their opinions and comment/reply to others about the learning topics using both the asynchronous online discussion platform and offline interactions. The data collected in this study was from the blended course, including online discussions and face-to-face lectures. All the online discussions over the whole session were recorded automatically, with offline interactions being recorded by videos. In these interaction records, online records accounted for 71.43%, and the rest was offline. Coding scheme for interactions and sentiments It is a prerequisite to be acquainted with sentiments and interactions with the coding schema. To examine the interactions in a learning process, interaction analysis proposed by Gunawardena et al. (1997) were Australasian Journal of Educational Technology, 2021, 37(2). 86 used as a coding scheme for online interactions. Five types of interactions were identified in the process of knowledge construction. Some interaction messages related only to emotional communication (e.g., greeting, cheering, kidding) without a direct relation to a learning task. Such interactions were called social- emotional interaction for building social bonds among students (Kwon et al., 2014). Accordingly, in this study, social-emotional interaction was deemed to be a new dimension, namely i6. Table 1 details the interaction coding scheme. Table 1 The coding scheme for interaction Interaction level Interaction Example Surface Sharing/comparing information (i1) Similar to your view, I also find resource associated with the AR technology. Discovery and exploration of dissonance (i2) l found that the applications of technology in education are not all educational technology. Deep Negotiation (i3) I think we can understand the definitions by the process of designing the learning materials. Test and modification (i4) With regard to the proposed solution, we can test the learning effectiveness and improve the weakness. Agreement statement (i5) According to the views and suggestions of all, I design a novel teaching framework. Social-emotion Social-emotional interaction (i6) Thank you for helping me solve the use of teaching tools. The six-dimension sentiment categories arising from Harris et al. (2014), positive, negative, neutral, confused, insightful, and joking, were used to examine how sentiments vary in online discussions. Previous studies have indicated that these six categories are sufficient to identify sentiments in conversational text (Harris et al., 2014). Furthermore, Harris et al. (2014) affirmed that sentiment classification was effective for examining textual data in online learning situations. Zheng and Huang (2016) used the aforementioned sentiment classification and confirmed it worked well in the Chinese context. Table 2 details the description of learning sentiments. Table 2 The coding scheme for learning sentiments Sentiment Description Example Positive (e1) Supporting or liking an opinion This resource is essential to understand educational technology. Negative (e2) Opposing or disliking the topic As for the teaching tool, I think it is not the best way to promote learning reflection. Neutral (e3) Ambivalent, or unclear user’s sentiment Mobile devices are used in learning environments. Insightful (e4) Conveying some innovative opinions or reflective thoughts In implementing the teaching process, I think it is important to get rid of the blind use of technology. Confused (e5) Transmitting opinions expressing bewilderment or perplexed feelings Pertaining to the application of technology, I am confused about the proportion of technology in education. Joking (e6) Meaning a user is only kidding My grandma cares what I eat not learn. Haha, Biological grandma. Overall, the blended course consisted of seven learning activities with a total of 1658 messages posted by the students. Using the above two coding schemes, two independent researchers coded a total of 500 discussion messages. Each message associated with a task was coded based on its temporal order. Two independent coders who have rich coding experience in online discussions were well trained to segment the discussion messages. Any discrepancies in the coding were discussed in person and resolved via consensus between coders and at least one of the study authors. These coded sentiments and interactions were treated as the test set for the text mining algorithms. To ensure the validity of this coding scheme, two experts were invited to check and verify the feasibility of the coding schemes, corresponding definitions Australasian Journal of Educational Technology, 2021, 37(2). 87 and examples. The kappa values of the seven learning activities were 0.89, 0.87, 0.90, 0,88, 0.91, 0.89 and 0.90, respectively, which indicated that the analysis was highly reliable. Data analysis To investigating the sentiment evolution with different interaction levels, a combined method of text mining and EAN was used to analyse the discussion messages in the blended learning context. The analysis process depicted in Figure 2 involves two steps: LSTM of text mining and ENA. Figure 2. The process of analysing discussion messages from the blended course Text mining This study used LSTM for text mining. LSTM is a widely used and powerful algorithm for extracting the sequential features of diverse sentiments and interactions (Chatterjee et al., 2019). Due to insufficient data from the blended course, the discussion messages from this study together with the public dataset, WorldUC (Zhou et al., 2018), were employed to train the LSTM and construct an optimal algorithm for classifying sentiments and interactions. After the initial training and tests using the WorldUC dataset, the 500 coded messages were fed into the optimal algorithm to train and test again. LSTM was used to extract the sequential features from the data, and SVM and AdaBoost were used to verify the results. LSTM, SVM and AdaBoost models were implemented using the scikit-learn Python library (http://scikit-learn.org/). The entire data was split into two datasets: the training dataset (90% of the entire data), and the test dataset (10% of the entire data). Ten-fold cross-validation was used for training LSTM, SVM and AdaBoost. The trained models were subsequently tested using the public and coded datasets after the completion of the cross-validation. To evaluate the performance of LSTM, this study used the three metrics of precision, recall, and F1 as used by Xie et al. (2018) (Table 3). As reported in Table 3, the results show that LSTM outperformed SVM and AdaBoost. This demonstrates that LSTM with time-sequential information is able to make use of the features of sentiments and interactions during the different stages of blended learning. Similarly, LSTM recognised six categories of interactions, with good performance. Table 3 The comparison results among three methods of LSTM, SVM and AdaBoost Interactions Sentiments SVM AdaBoost LSTM SVM AdaBoost LSTM Cross- validation Precision 0.623 0.749 0.801 0.799 0.813 0.857 Recall 0.598 0.672 0.712 0.653 0.568 0.525 F1 0.610 0.708 0.754 0.719 0.669 0.651 Test Precision 0.532 0.721 0.831 0.737 0.873 0.893 Recall 0.652 0.578 0.693 0.712 0.795 0.764 F1 0.586 0.642 0.756 0.724 0.832 0.823 Australasian Journal of Educational Technology, 2021, 37(2). 88 Epistemic network analysis (ENA) This study to identify the evolution of sentiments and interactions in blended courses using longitudinal data commenced on September 2017 and ended on December 2017. As interactions in the blended learning course increased, the size of the social network grew over time. According to the method of stage division used in the previous studies (Snijders et al., 2010; Zhang et al., 2016), the different stages were identified in the learning process to explore the evolution of sentiments. This study divided the learning process into five stages labelled 1 to 5. To measure the stability of five stages, the Jaccard coefficients were calculated and varied from 0.538 to 0.912 between two sequential periods (Snijders et al., 2010), revealing that the network changes of five stages were smooth enough and appropriate for this study. For this longitudinal blended course of five stages, this study employed ENA to characterise sentiment evolution with dynamic interaction levels. As previously mentioned, ENA contributes to revealing the dynamic network connections of sentiments in blended learning discussions. Essentially, ENA measures the relationships between the six types of sentiments by quantifying their co-occurrences. Before applying ENA, the qualitative text data of the discussion messages were converted into quantitative data. Using this classified data, the dataset was processed according to the indicators (Table 4), comprising five stages of blended learning from 1 to 5, three types of interaction levels, namely surface, deep, and social-emotion, and six-dimensional sentiments (i.e., positive, neutral, negative, confuse, insightful, and joking), detailed in Table 2. Initially, the indicators of sentiments and interactions were annotated 1 and 0 for presence and absence, respectively. As shown in Table 4, each row represents a segment of ENA, while the units of ENA are comprised of every stage. Table 4 Schematic design example of the data processing for ENA Stage Interaction level Interaction Sentiment i1 i2 i3 i4 i5 i6 e1 e2 e3 e4 e5 e6 1 Surface 1 0 0 0 0 0 1 0 0 0 0 0 2 Deep 0 0 0 0 1 0 0 0 0 0 1 0 3 Deep 0 0 1 0 0 0 0 0 1 0 0 0 3 Social-emotion 0 0 0 0 0 1 0 0 0 0 0 1 4 Deep 0 0 0 1 0 0 0 0 0 1 0 0 5 Surface 0 1 0 0 0 0 0 1 0 0 0 0 Results Sentiment distribution with different interaction levels by text mining To further examine the sentiments in the five stages in the blended learning context, a descriptive analysis of students’ sentiments was conducted, with the results shown in Figure 3. Figure 3 shows that the frequency of sentiments was least at stage 1, especially for social-emotion interactions. In stage 1, there were fewer negative sentiments in the surface interactions than in the other four stages. As the learning progressed, all types of sentiments expressed by students increased at stages 2 and 3. Particularly in stages 2 and 3, social-emotion interactions evoked more diverse sentiments than in the initial period. However, each interaction level includes confused sentiments, even in stages 4 and 5. However, compared with stages 2 and 3, there was a slight decrease in confused sentiments in stages 4 and 5 and tended to stabilise. Australasian Journal of Educational Technology, 2021, 37(2). 89 Figure 3. Sentiment distribution of different interaction levels for the five stages Sentiment evolution with interaction levels by ENA To explore the differences in sentiments in interactions across the five stages, this study further conducted a series of ENA. Figure 4 shows the ENA network of the relationships among the six types of sentiments. Within the cartesian coordinate system in Figure 4, the positive (i.e., positive, insightful) and joking sentiments are mainly in quadrants I and II, while the negative (i.e., negative and confused sentiments) sentiments are in quadrants III and IV. Roughly, the X-axis distinguishes between positive and joking sentiments, whereas the Y-axis differentiates between positive and negative sentiments. Figure 4. Comparison results of sentiment connections for surface (red), deep (blue) and social-emotion (orange) interaction levels ENA analyses all the chronological networks simultaneously so that they can be compared visually and statistically. Table 5 details the differences model between the three levels of surface, deep and social- emotion interactions. As illustrated in Figure 4, negative sentiments are moderately associated with several other sentiments such as joking, confused and neutral sentiments. In the difference models, the red line shows that surface interactions might be more likely to link negative and confused sentiments, leading to the curiosity of solving problem. The blue lines connecting confused, positive and insightful sentiments suggest that deep interactions connect the student’s confusion to insightful solutions. These differences between the three networks can be seen more clearly in the sub-network graph shown in Figure 5. Australasian Journal of Educational Technology, 2021, 37(2). 90 Table 5 Comparison results for the three levels of surface, deep and social-emotion interactions Comparison group Mean SD T value and p Effect size X Y X Y X Y X Y Surface Deep -0.4 -0.85 -0.75 0.59 0.44 0.09 0.66 0.47 t(4.35) = 2.23 p = 0.08 t(7.21) = 3.70 p = 0.01* d = 1.41 d = 2.34 Surface Social-emotion -0.401.25 -0.75 0.16 0.44 1.00 0.66 0.99 t(5.51) = -3.37 p = 0.02* t(7.00) = 1.70 p = 0.13 d=2.13 d = 1.08 Deep Social-emotion -0.85 1.25 0.59 0.16 0.09 1.00 0.47 0.99 t(4.07) = -4.68 p = 0.01* t(5.73) = -0.90 p = 0.40 d = 2.73 d = 0.19 The perimeter boxes in Figure 5 are around the mean locations representing the 95% confidence intervals. It can be seen that the sentiments in the three groups are grouped along the X-axis and Y-axis. The diagram for surface interactions has relatively weak connections between the confused to positive/insightful/joking sentiments, whereas significant associations are found among positive, neutral, and negative sentiments. For the deep interactions, the strongest connections are on the left side, with relatively stronger connections between insightful-positive, negative-insightful, and negative-positive sentiments. In contrast, the network for the social-emotion interactions shows the stronger connections in joking-positive and joking-negative sentiments. Figure 5. Sentiment change network for different interaction levels Sentiment evolution during different periods by ENA An additional ENA was performed to characterise sentiment evolution over the five stages. Based on the results shown in Figures 3, 4 and 5, sentiment evolution over the five stages can be identified along with interactions. As suggested by Huang et al. (2019), the learning process might be divided into three periods, the initial period, the collision and sublimation period, and the stable period. The ENA results from different stages are visualised using network graphs where the nodes represent the six types of sentiments: the edges reflect the relative frequency of co-occurrence, or the connection, between two codes. In particular, the nodes of the network in Figure 6 represent the constructs (six types of sentiments) and the weight of the thickness of edges indicates the strength of their connection. Australasian Journal of Educational Technology, 2021, 37(2). 91 Figure 6. Sentiment evolution for the five stages of blended learning contexts The findings reveal that there was a joint connection between positive-insightful-neutral sentiments at all stages. The changes of co-occurrence sentiments indicated a learning diagram over the three periods. In stage 1 (initial period), the negative-joking connection was the strongest while some of other connections focus on positive-negative connections (e.g., positive-negative, positive-confused). Stronger occurrence relationships were found in the collision and sublimation period for positive-neutral, negative-insightful, negative-confused, and negative-joking connections. These co-occurrent connections indicate that students might experience negative and confused sentiments while solving problems during blended learning activities. In turn, these sentiments tend to evoke joking, insightful and positive sentiments. The above results overall reflect students’ blended learning experiences, by highlighting sentiment evolution from positive to confused/negative to insightful sentiments in the process of constructing knowledge. Discussion Sentiment changes with different interaction levels This study revealed a strong tendency for sentiment to change with surface, deep and social-emotion interactions. Students expressed their curiosity sentiments evoked by surface interactions, whereas deep interactions promoted the problem-solving process, in which sentiments changed from confusion/negative to insightful/positive sentiments. That is, deep interactions can play an important role in improving problem-solving abilities, and when problems are settled, sentiments shift from negative/confused to positive/insightful sentiments. Aligning with Qin et al. (2014), negative sentiments might be conducive to promoting deep interactions for solving problems. As noted by Kort et al. (2001), sentiments expressed by students might vary from positive to negative sentiments, and from surface to deep interactions. This finding is opposite to the view of Yang et al. (2016) who found it was less likely that learning-related content would be discussed by diverse interactions if students experienced confused sentiments. However, there is a need for additional studies to identify the proper level of confused sentiments to facilitate the learning process. In addition, the results from this study suggested that social-emotion interactions can alleviate the influence caused by confused sentiments when completing learning activities. In this regard, social-emotion interactions triggered joking sentiments. This finding suggests that there is a tendency for joking sentiments to adjust the atmosphere caused by negative and confused sentiments, which is consistent with the finding of Manwaring et al. (2017). However, as indicated by Huang et al. (2019), joking sentiments are often negatively associated with learning performance. Therefore, further studies could examine the optimal extent to joking sentiments for constructing knowledge. Australasian Journal of Educational Technology, 2021, 37(2). 92 Sentiment evolution with different learning stages of blended learning In terms of sentiment evolution in blended learning, ENA models showed a significant and positive tendency (e.g., positive sentiments) on sentiment formation at the beginning of blended learning. As the learning process progressed, the findings from ENA showed that the connections among positive-confused, negative-insightful, negative-joking and positive-joking changed significantly with the increase in blended learning activities over time (Garcia et al., 2016; Pérez-López et al., 2020). As suggested by Huang et al. (2019), the periods of sentiment evolution were divided into the initial, collision and sublimation, and stable periods. The three periods were characterised by the specific diagrams of the latent network structure of sentiment changes in this study. As such, it is possible to give impetus to the collision and sublimation period by guiding students to experience slightly negative and confused sentiments for deep interactions at the beginning of blended learning. A progressive learning mode was identified in blended discussions according to Huang et al. (2019), suggesting that sentiments start from positive to negative to insightful or positive, along with interactions from surface to deep. Nevertheless, the three periods of sentiment evolution could be only viewed as an initial investigation. Further studies need to examine the robustness of the three periods. Conclusions This research aimed to elucidate how sentiments evolved over different interaction levels at different stages of blended learning. To do this, this study combined the methods of text mining and ENA. From the qualitative data that was recorded during the discussions in the blended learning activities, the combined method discovered six types of sentiments and six-dimensional interactions, together with the five learning stages identified by LSTM. The change networks by ENA reflected the insights into sentiment evolution with interaction levels across the five stages of blended learning. This study showcased an initial exploration of how the combined use of learning analytics and ENA can advance sentiment analysis in blended learning. Moreover, the results demonstrated a complementary contribution to the findings that could not be obtained if either ENA or LSTM was applied alone. The findings of this study should be interpreted with caution. The data is from a single course. This might negatively affect the generalisation of the results. Also, there are some limitations to this preliminary exploration of learning analytics. To address these limitations, our future studies will perform the same analysis using the data from large-scale blended courses. Although care has been taken to ensure that the methodology in this study is sound, other methods such as questionnaires and interviews can be used to explore sentiment evolution with different interaction levels. Acknowledgments This work was supported by the National Natural Science Foundation of China (No. 62037001, 62007031 and 61877020), and the Key Research and Development Program of Zhejiang Province (No. 2021C03141). References Asarta, C. J., & Schmidt, J. R. (2020). The effects of online and blended experience on outcomes in a blended learning environment. The Internet and Higher Education, 44, 100708. https://doi.org/10.1016/j.iheduc.2019.100708 Boelens, R., De Wever, B., & Voet, M. (2017). Four key challenges to the design of blended learning: A systematic literature review. Educational Research Review, 22, 1-18. https://doi.org/10.1016/j.edurev.2017.06.001 Bressler, D. M., Bodzin, A. M., Eagan, B., & Tabatabai, S. (2019). Using epistemic network analysis to examine discourse and scientific practice during a collaborative game. Journal of Science Education and Technology, 28(5), 553-566. https://doi.org/10.1007/s10956-019-09786-8 Cabrera, L. Y., Fitz, N. S., & Reiner, P. B. (2015). Empirical support for the moral salience of the therapy-enhancement distinction in the debate over cognitive, affective and social enhancement. Neuroethics, 8(3), 243-256. https://doi.org/10.1007/s12152-014-9223-2 https://doi.org/10.1016/j.iheduc.2019.100708 https://doi.org/10.1016/j.edurev.2017.06.001 https://doi.org/10.1007/s10956-019-09786-8 https://doi.org/10.1007/s12152-014-9223-2 Australasian Journal of Educational Technology, 2021, 37(2). 93 Celestial-Valderama, A. M., Vinluan, A., & Moraga, S. D. (2021). Mining students’ feedback in a general education course: Basis for improving blended learning implementation. International Journal of Computing Sciences Research, 5(1), 568-583. https://www.stepacademic.net/ijcsr/article/view/214 Chatterjee, A., Gupta, U., Chinnakotla, M. K., Srikanth, R., Galley, M., & Agrawal, P. (2019). Understanding emotions in text using deep learning and big data. Computers in Human Behavior, 93, 309-317. https://doi.org/10.1016/j.chb.2018.12.029 Cocquyt, C., Zhu, C., Diep, A. N., De Greef, M., & Vanwing, T. (2019). Examining the role of learning support in blended learning for adults' social inclusion and social capital. Computers & Education, 142, 103610. https://doi.org/10.1016/j.compedu.2019.103610 Csanadi, A., Eagan, B., Kollar, I., Shaffer, D. W., & Fischer, F. (2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13(4), 419-438. https://doi.org/10.1007/s11412-018-9292-z Garcia, D., Kappas, A., Küster, D., & Schweitzer, F. (2016). The dynamics of emotions in online interaction. Royal Society Open Science, 3(8), 160059. https://doi.org/10.1098/rsos.160059 Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431. https://hdl.handle.net/2149/772 Han, F., & Ellis, R. A. (2019). Identifying consistent patterns of quality learning discussions in blended learning. The Internet and Higher Education, 40, 12-19. https://doi.org/10.1016/j.iheduc.2018.09.002 Han, Z., Huang, C., Huang, Q., & Yu, J. (2020). Sentiment evolutions in blended learning contexts: Investigating dynamic interactions using simulation investigation for empirical social network analysis. Proceedings of the International Conference on Blended Learning (pp. 249-263). Springer. https://link.springer.com/chapter/10.1007/978-3-030-51968-1_21 Harris, S. C., Zheng, L., & Kumar, V. (2014). Multi-dimensional sentiment classification in online learning environment. Proceedings of the 6th International Conference on Technology for Education (pp. 172-175). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/T4E.2014.50 Hew, K. F., Hu, X., Qiao, C., & Tang, Y. (2020). What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach. Computers & Education, 145, 103724. https://doi.org/10.1016/j.compedu.2019.103724 Hod, Y., Katz, S., & Eagan, B. (2020). Refining qualitative ethnographies using epistemic network analysis: A study of socioemotional learning dimensions in a humanistic knowledge building community. Computers & Education, 156, 103943. https://doi.org/10.1016/j.compedu.2020.103943 Huang, C. Q., Han, Z. M., Li, M. X., Jong, S. Y., & Tsai, C. C. (2019). Investigating students' interaction patterns and dynamic learning sentiments in online discussions. Computers & Education, 140, 103589. https://doi.org/10.1016/j.compedu.2019.05.015 Kort, B., Reilly, R., & Picard, R. W. (2001). An affective model of interplay between emotions and learning: Reengineering educational pedagogy-building a learning companion. Proceedings of the International Conference on Advanced Learning Technologies (pp. 43-46). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICALT.2001.943850 Kurucay, M., & Inan, F. A. (2017). Examining the effects of learner-learner interactions on satisfaction and learning in an online undergraduate course. Computers & Education, 115, 20-37. https://doi.org/10.1016/j.compedu.2017.06.010 Kwon, K., Liu, Y. H., & Johnson, L. P. (2014). Group regulation and social-emotional interactions observed in computer supported collaborative learning: Comparison between good vs. poor collaborators. Computers & Education, 78, 185-200. https://doi.org/10.1016/j.compedu.2014.06.004 Law, K. M., Geng, S., & Li, T. (2019). Student enrolment, motivation and learning performance in a blended learning environment: The mediating effects of social, teaching, and cognitive presence. Computers & Education, 136, 1-12. https://doi.org/10.1016/j.compedu.2019.02.021 Liang, G., On, B.-W., Jeong, D., Kim, H.-C., & Choi, G. S. (2018). Automated essay scoring: A Siamese bidirectional LSTM neural network architecture. Symmetry, 10(12), 682. https://doi.org/10.3390/sym10120682 Liu, R., & Shi, C. (2018). Exploring different types of interaction on collaborative learning in online platforms. International Journal of Innovation and Learning, 23(4), 386-399. https://doi.org/10.1504/IJIL.2018.092040 https://www.stepacademic.net/ijcsr/article/view/214 https://doi.org/10.1016/j.chb.2018.12.029 https://doi.org/10.1016/j.compedu.2019.103610 https://doi.org/10.1007/s11412-018-9292-z https://doi.org/10.1098/rsos.160059 https://hdl.handle.net/2149/772 https://doi.org/10.1016/j.iheduc.2018.09.002 https://link.springer.com/chapter/10.1007/978-3-030-51968-1_21 https://doi.org/10.1109/T4E.2014.50 https://doi.org/10.1016/j.compedu.2019.103724 https://doi.org/10.1016/j.compedu.2020.103943 https://doi.org/10.1016/j.compedu.2019.05.015 https://doi.org/10.1109/ICALT.2001.943850 https://doi.org/10.1016/j.compedu.2017.06.010 https://doi.org/10.1016/j.compedu.2014.06.004 https://doi.org/10.1016/j.compedu.2019.02.021 https://doi.org/10.3390/sym10120682 https://doi.org/10.1504/IJIL.2018.092040 Australasian Journal of Educational Technology, 2021, 37(2). 94 Liu, Z., Yang, C., Rüdian, S., Liu, S., Zhao, L., & Wang, T. (2019). Temporal emotion-aspect modeling for discovering what students are concerned about in online course forums. Interactive Learning Environments, 27(5-6), 598-627. https://doi.org/10.1080/10494820.2019.1610449 Manwaring, K. C., Larsen, R., Graham, C. R., Henrie, C. R., & Halverson, L. R. (2017). Investigating student engagement in blended learning settings using experience sampling and structural equation modeling. The Internet and Higher Education, 35, 21-33. https://doi.org/10.1016/j.iheduc.2017.06.002 Marchand, G. C., & Gutierrez, A. P. (2012). The role of emotion in the learning process: Comparisons between online and face-to-face learning settings. The Internet and Higher Education, 15(3), 150-160. https://doi.org/10.1016/j.iheduc.2011.10.001 Misiejuk, K., Wasson, B., & Egelandsdal, K. (2021). Using learning analytics to understand student perceptions of peer feedback. Computers in Human Behavior, 117, 106658. https://doi.org/10.1016/j.chb.2020.106658 Muñoz-Cristóbal, J. A., Hernández-Leo, D., Carvalho, L., Martinez-Maldonado, R., Thompson, K., Wardak, D., & Goodyear, P. (2018). 4FAD: A framework for mapping the evolution of artefacts in the learning design process. Australasian Journal of Educational Technology, 34(2), 16-34. https://doi.org/10.14742/ajet.3706 Ortigosa, A., Martín, J. M., & Carro, R. M. (2014). Sentiment analysis in Facebook and its application to e-learning. Computers in Human Behavior, 31, 527-541. https://doi.org/10.1016/j.chb.2013.05.024 Owston, R., York, D. N., & Malhotra, T. (2019). Blended learning in large enrolment courses: Student perceptions across four different instructional models. Australasian Journal of Educational Technology, 35(5), 29-45. https://doi.org/10.14742/ajet.4310 Paulus, T. M., & Wise, A. F. (2019). Looking for insight, transformation, and learning in online talk. Routledge. https://doi.org/10.4324/9781315283258 Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315-341. https://doi.org/10.1007/s10648-006-9029-9 Pérez-López, R., Gurrea-Sarasa, R., Herrando, C., Martín-De Hoyos, M. J., Bordonaba-Juste, V., & Utrillas-Acerete, A. (2020). The generation of student engagement as a cognition-affect-behaviour process in a Twitter learning experience. Australasian Journal of Educational Technology, 36(3), 132-146. https://doi.org/10.14742/ajet.5751 Purarjomandlangrudi, A., & Chen, D. (2020). Exploring the influence of learners’ personal traits and perceived course characteristics on online interaction and engagement. Educational Technology Research and Development, 68(5): 2635-2657. https://doi.org/10.1007/s11423-020-09792-3 Qin, J., Zheng, Q., & Li, H. (2014). A Study of learner-oriented negative emotion compensation in e- learning. Journal of Educational Technology & Society, 17(4), 420-431. https://www.jstor.org/stable/jeductechsoci.17.4.420 Rasheed, R. A., Kamsin, A., & Abdullah, N. A. (2020). Challenges in the online component of blended learning: A systematic review. Computers & Education, 144, 103701. https://doi.org/10.1016/j.compedu.2019.103701 Rolim, V., Ferreira, R., Lins, R. D., & Gǎsević, D. (2019). A network-based analytic approach to uncovering the relationship between social and cognitive presences in communities of inquiry. The Internet and Higher Education, 42, 53-65. https://doi.org/10.1016/j.iheduc.2019.05.001 Rubenking, B. (2019). Emotion, attitudes, norms and sources: Exploring sharing intent of disgusting online videos. Computers in Human Behavior, 96, 63-71. https://doi.org/10.1016/j.chb.2019.02.011 Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9-45. https://doi.org/10.18608/jla.2016.33.3 Shu, H., & Gu, X. (2018). Determining the differences between online and face-to-face student–group interactions in a blended learning course. The Internet and Higher Education, 39, 13-21. https://doi.org/10.1016/j.iheduc.2018.05.003 Simmons, M., Colville, D., Bullock, S., Willems, J., Macado, M., McArdle, A., Tare, M., Kelly, J., Taher, M. A., & Middleton, S. (2020). Introducing the flip: A mixed method approach to gauge student and staff perceptions on the introduction of flipped pedagogy in pre-clinical medical education. Australasian Journal of Educational Technology, 36(3), 163-175. https://doi.org/10.14742/ajet.5600 Snijders, T. A., Van de Bunt, G. G., & Steglich, C. E. (2010). Introduction to stochastic actor-based models for network dynamics. Social Networks, 32(1), 44-60. https://doi.org/10.1016/j.socnet.2009.02.004 https://doi.org/10.1080/10494820.2019.1610449 https://doi.org/10.1016/j.iheduc.2017.06.002 https://doi.org/10.1016/j.iheduc.2011.10.001 https://doi.org/10.1016/j.chb.2020.106658 https://doi.org/10.14742/ajet.3706 https://doi.org/10.1016/j.chb.2013.05.024 https://doi.org/10.14742/ajet.4310 https://doi.org/10.4324/9781315283258 https://doi.org/10.1007/s10648-006-9029-9 https://doi.org/10.14742/ajet.5751 https://doi.org/10.1007/s11423-020-09792-3 https://www.jstor.org/stable/jeductechsoci.17.4.420 https://doi.org/10.1016/j.compedu.2019.103701 https://doi.org/10.1016/j.iheduc.2019.05.001 https://doi.org/10.1016/j.chb.2019.02.011 https://doi.org/10.18608/jla.2016.33.3 https://doi.org/10.1016/j.iheduc.2018.05.003 https://doi.org/10.14742/ajet.5600 https://doi.org/10.1016/j.socnet.2009.02.004 Australasian Journal of Educational Technology, 2021, 37(2). 95 Tang, S., Peterson, J. C., & Pardos, Z. A. (2016). Deep neural networks and how they apply to sequential education data. Proceedings of the 3rd International Conference on Learning Scale (pp. 321-324). Association for Computing Machinery. https://doi.org/10.1145/2876034.2893444 Wang, M., Guo, W., Le, H., & Qiao, B. (2020). Reply to which post? An analysis of peer reviews in a high school SPOC. Interactive Learning Environments, 28(5), 574-585. https://doi.org/10.1080/10494820.2019.1696840 Wang, Z., Anderson, T., Chen, L., & Barbera, E. (2017). Interaction pattern analysis in cMOOCs based on the connectivist interaction and engagement framework. British Journal of Educational Technology, 48(2), 683-699. https://doi.org/10.1111/bjet.12433 Wang, Z., Chen, L., & Anderson, T. (2014). A framework for interaction and cognitive engagement in connectivist learning contexts. International Review of Research in Open and Distributed Learning, 15(2), 121-141. https://doi.org/10.19173/irrodl.v15i2.1709 Wei, X., Lin, H., Yang, L., & Yu, Y. (2017). A convolution-LSTM-based deep neural network for cross- domain MOOC forum post classification. Information, 8(3), 92. https://doi.org/10.3390/info8030092 Wu, L., Liu, Q., Mao, G., & Zhang, S. (2020). Using epistemic network analysis and self-reported reflections to explore students' metacognition differences in collaborative learning. Learning and Individual Differences, 82, 101913. https://doi.org/10.1016/j.lindif.2020.101913 Xie, K., Di Tosto, G., Lu, L., & Cho, Y. S. (2018). Detecting leadership in peer-moderated online collaborative learning through text mining and social network analysis. The Internet and Higher Education, 38, 9-17. https://doi.org/10.1016/j.iheduc.2018.04.002 Yang, D., Kraut, R., & Rosé, C. P. (2016). Exploring the effect of student confusion in massive open online courses. Journal of Educational Data Mining, 8(1), 52-83. https://doi.org/10.5281/zenodo.3554605 Yang, X., Li, J., & Xing, B. (2018). Behavioral patterns of knowledge construction in online cooperative translation activities. The Internet and Higher Education, 36, 13-21. https://doi.org/10.1016/j.iheduc.2017.08.003 Zhang, J., Skryabin, M., and Song, X. (2016). Understanding the dynamics of MOOC discussion forums with simulation investigation for empirical network analysis (SIENA). Distance Education, 37(3), 270-286. https://doi.org/10.1080/01587919.2016.1226230 Zheng, L., & Huang, R. (2016). The effects of sentiments and co-regulation on group performance in computer supported collaborative learning. The Internet and Higher Education, 28, 59-67. https://doi.org/10.1016/j.iheduc.2015.10.001 Zhou, Y., Huang, C., Hu, Q., Zhu, J., & Tang, Y. (2018). Personalized learning full-path recommendation model based on LSTM neural networks. Information Sciences, 444, 135-152. https://doi.org/10.1016/j.ins.2018.02.053 Corresponding author: Changqin Huang, cqhuang@zju.edu.cn Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC- ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY- NC-ND 4.0. Please cite as: Huang, C., Han, Z., Li, M., Wang, X., & Zhao, W. (2021). Sentiment evolution with interaction levels in blended learning environments: Using learning analytics and epistemic network analysis. Australasian Journal of Educational Technology, 37(2), 81-95. https://doi.org/10.14742/ajet.6749 https://doi.org/10.1145/2876034.2893444 https://doi.org/10.1080/10494820.2019.1696840 https://doi.org/10.1111/bjet.12433 https://doi.org/10.19173/irrodl.v15i2.1709 https://doi.org/10.3390/info8030092 https://doi.org/10.1016/j.lindif.2020.101913 https://doi.org/10.1016/j.iheduc.2018.04.002 https://doi.org/10.5281/zenodo.3554605 https://doi.org/10.1016/j.iheduc.2017.08.003 https://doi.org/10.1080/01587919.2016.1226230 https://doi.org/10.1016/j.iheduc.2015.10.001 https://doi.org/10.1016/j.ins.2018.02.053 https://creativecommons.org/licenses/by-nc-nd/4.0/ https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.6749