 > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 1 Abstract— Among the discussion-content analytical tools in the field of e-learning research, the community of inquiry (CoI) model is extensively applied and continuously improved by its users. This model investigates the types of elements that are manifested through inquiry-based learning processes in online discussions. They are social, cognitive, teaching and metacognitive presences. These elements are essential for meaningful student interactions to take place in online learning environments. In particular, the metacognitive presence construct of the CoI model discovers the students’ ability of self and co-regulation of learning in an online learning environment. However, the metacognitive presence construct of the CoI model has not been evaluated along with the other components of the model. Therefore, in this paper the CoI model was re-evaluated to determine its reliability in analysing discussions in online courses on information technology related subjects. The evaluation is conducted with four online courses designed and developed for a distance learning programme in Sri Lanka. The paper discusses the modifications that were needed to make the model more applicable for conducting discussion-content analysis in similar types of online learning environments, and reports on the results of the final evaluation. Furthermore, the findings of the study imply that the theoretical framework of the CoI model needs to be improved to properly enclose the metacognitive presence component. In spite of this, the study adds points to the CoI model supporting for its well applicability and reliability in analysing online discussion content in information technology related courses. Index Terms— inquiry-based learning, reliability, social presence, cognitive presence, teaching presence, metacognitive presence I. INTRODUCTION t has become a common practice at higher educational departments conducting distance learning programmes to use e-learning to deliver instructional materials to students. Other than the materials, an e-learning environment can provide students with platforms to engage in discussions. Online discussions based on forums can be created to support both place and time-independent communications, providing more opportunities for students to converse with other students and teachers. If a forum is kept open for discussion throughout a course then it can become a record of how students and teachers interacted with each other. At the completion of the course, this forum can serve as a rich source of information for the course coordinators and designers to analyse and understand student interactions during discussions. The content can be analysed using different types of analytical instruments, in order to study factors such as student participation and interaction, as well as cognitive, metacognitive and social cues (e.g. [1] and [2]), critical thinking (e.g. [3], [4], and [5]) and group development [6]. Analytical instruments are critically examined with respect to two parameters: validity and reliability. Reliability is a factor determining the quality of an analytical model [7]. Reliability of an analytical model is measured with statistical techniques such as percentage agreement, Cohen‘s Kappa or Pearson‘s correlation [8], [9]. A reliable analytical model strengthens the validity of the results in the content analysis. Therefore, results of a discussion content analysis should be preceded by an assessment of reliability of the analytical model. Since most of the issues associated with validity and reliability can be mitigated with sound analytical models having ―discrete categories, and clear indicators‖ [10, p. 2], analysts should try to improve analytical models by appropriately modifying their lists and the definitions of categories and indicators. Among the instruments used for online discussion content analysis, the model based on the Community of Inquiry (CoI) framework is extensively used (e.g. [11] and [12]) in the field of online learning, and it has been continually refined and adapted by researchers [13]. The CoI framework mainly focuses on the nature of educational transaction in an online learning environment, and it has emerged in the context of asynchronous text-based online discussions [14]. The framework reflects the critical thinking processes that could exist in an online discussion. The analytical tool based on this framework is named the Community of Inquiry (CoI) model [15], [16]. It can be used to identify social, cognitive, teaching and metacognitive presence in online discussions. While social, cognitive and teaching presence are essential to foster meaningful interactions, cognitive presence is considered to be the most important element to identify critical thinking activities in a CoI [4]. Metacognitive presence elements are closely linked to critical thinking and higher learning in a discussion [16]. Reviewing research on CoI framework and its analytical model, Garrison and Arbaugh [15] report that there is a need of evaluating the model with all its components and further studying the relationships between the components of the CoI theoretical framework. Moreover, they encourage future research to evaluate the CoI model and the framework in disciplines other than education. As an attempt to meet this requirement, Shea et al. [17] have examined the CoI model – excluding metacognitive presence component, with minor modifications. However, their findings also highlight the necessity of improving the analytical model further, in particular, to enhance the reliability of the social presence construct. In spite of that, a study of Akyol and Garrison [16] has introduced a new component named ‗metacognitive presence‘, to the CoI model which has not been evaluated with the rest of the other components of the CoI model. Metacognitive skill is considered essential for students learning in distance/ online learning environments [18]. Studying metacognitive presence in online discussions can reveal students‘ latent knowledge and regulatory skills further enabling researchers to discern student learning processes in online learning environments [16]. This understanding can help to design self-directed online learning activities for distance learning programmes. Therefore, we decided to re-evaluate the CoI model with its metacognitive presence construct. This paper reports on our research to re-evaluate and attempts to improve the CoI model with its four components: social, cognitive, teaching and metacognitive presence to make it more reliable for analysing forum discussions in online courses. Re-evaluation of community of inquiry model with its metacognitive presence construct T. A. Weerasinghe, K. P. Hewagamage, and R. Ramberg I > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 2 A. Purpose The CoI model has become a useful tool for investigating students‘ interactions in inquiry based online discussions [19], [17]. However, it needs further improvements to make it a more reliable tool to investigate students‘ learning in online discussions [20]. The model and the framework have further not been examined with its new component, the metacognitive presence construct. The new construct has a potential for disclosing latent knowledge and regulatory skills which cannot be identified with the cognitive presence construct alone. Therefore, we wish to determine whether we can reliably use the metacognitive construct along with the rest of the components of the CoI model. For this purpose, we evaluate the CoI model by analysing a set of discussions selected from four online courses. B. Theoretical Framework and Model From its inception, the community of inquiry (CoI) framework has had three components enclosing three types of elements essential for meaningful interactions to take place in a community [4]. The three types of elements and the components are named social, cognitive and teaching presences. The teaching presence component represents the role of the teacher, which is often carried out with the collaboration of a number of individuals who are not teachers [21]. In [22], Shea and Bidjerano recommended that this framework should consider the online learner‘s self-regulatory learning behaviour and self-efficacy. In line with this reasoning, they suggested adding to the framework another element: learner presence. This stood independently from the rest of the components of the framework. Akyol and Garrison, however, did not agree with this modification reporting that ―creation of a learning presence construct would implicitly assign teaching presence to only that of the teacher‖ and it ―is incongruent with the premise of a community of inquiry‖ [16, p. 189]. Instead, they introduced the metacognitive presence component, which was placed at the intersection of the teaching and cognitive presence of the CoI framework (see Fig1). Fig1. Community of Inquiry (CoI) Framework (Adapted from [16]) In accordance with this framework, the CoI model has four coding schemes: social, cognitive, teaching and metacognitive presence. Social presence is defined as ―the ability of learners to project themselves socially and emotionally in a CoI‖ [4, p. 94]. The social presence coding scheme has three categories: affective, open communication and group cohesion. Garrison, Anderson and Archer define these categories respectively ―in terms of the participants identifying with the community, communicating purposefully in a trusting environment and developing interpersonal relationships‖ [14, p. 7]. Cognitive presence is the main component of the CoI framework. It describes ―the extent to which learners are able to construct and confirm meaning through sustained discourse in a critical community of inquiry‖ [23, p. 1]). The cognitive presence coding scheme has four categories that represent the phases of an inquiry process in a collaborative learning environment. The phases are triggering event, exploration, integration and resolution. Triggering event is the initiation phase of a critical inquiry where an issue, dilemma or problem is identified or recognised. The next phase is ‗exploration‘ where learners tend to grasp the nature of the problem and move to explore relevant information. In the integration phase learners construct meaning from the ideas generated in the exploratory phase. The last phase of the critical inquiry model is ‗resolution‘. It indicates a resolution of dilemma or problem that caused the triggering event. Teaching presence is defined as ―the design, facilitation and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes‖ [21, p. 5]. Teaching presence represents the role of teaching, which is carried out by the collaborative involvement of participants in a community [21]. This component of the analytical tool has three categories: design and organisation, facilitating discourse and direct instruction. The design and organisation category considers the role of a teacher during the designing and planning process of online learning activities, while the other two categories – facilitating discourse and direct instruction – investigate signs of teaching presence during students‘ engagement in learning activities. Metacognitive presence is the new component that Akyol and Garrison [16] introduced to the CoI model. Referring to contemporary research into metacognition and learning (e.g. [24] and [25]), they defined metacognition in an online learning community as ―the set of higher knowledge and skills to monitor and regulate manifest cognitive processes of self and others‖ [16, p. 184]. Moreover, motivational states for learning are considered in describing metacognitive presence in online discussions. The component has three categories: knowledge of cognition (KC), monitoring of cognition (MC) and regulation of cognition (RC). According to [16], KC ―refers to awareness of self as a learner in a broad sense‖, where ―knowledge includes entering knowledge and motivation associated with the inquiry process, academic discipline, and expectancies‖ (p. 184). KC characterises pre-task metacognitive states; in other terms, more general aspects of metacognition observed at anytime. In comparison, MC and RC represent activity-based SOCIAL PRESENC E COGNITIV E PRESENC E TEACHING PRESENCE Setting Climate Support ing Discour se Monitorin g & Regulatio n EDUCATIONA L EXPERIENCE METACOG NITIVE PRESENCE > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 3 metacognitive states, which can be observed during the learning process. II. CONTEXT The examination of the CoI framework and model was carried out by analysing eight discussion threads in the online learning environment of the BIT (Bachelor of Information Technology) degree programme (www.bit.lk) at the University of Colombo School of Computing (UCSC), Sri Lanka. The students of the degree programme did not receive any lectures or feedback from the university teachers on site or in a physical lecture room. Instead, the university distributed course materials using a virtual learning environment. The online courses were designed and developed by the teams of content developers and instructional designers at the e- Learning Centre of the UCSC. There were discussion forums for each course, which provided an opportunity for the students to discuss their concerns. A facilitator was there to assist students in the forums to find answers to their problems. The courses were offered in English and the recommended language to be used in the online discussions was English. Even though English language competency was considered as an entry requirement for the BIT programme, this was not the first language of the majority of the students. Therefore, their understandings of others‘ responses were different from native English speakers and the expressions were biased towards their mother tongue. III. METHOD The model was examined by evaluating it twice. The first evaluation was conducted using the CoI model improved by Shea et al. [17]. This model is composed of three separate coding schemes for investigating indications of social, cognitive and teaching presence. Along with these, we used a coding scheme for identifying signs of metacognitive presences. It was developed based on the classification of metacognitive presence elements in online discussions by Akyol and Garrison [16]. The first evaluation was carried out by analysing four discussion threads (Sample 1) randomly selected from four online courses in the BIT programme. The coding schemes were improved based on the comments of the coders and the issues that arose at the first evaluation. The improved coding schemes were re-evaluated by analysing eight online discussions that consisted of two online discussions from each of the four courses (see Table 1). These eight included the four discussion threads (Sample 1) that we previously analysed in the first evaluation and four other randomly selected discussion threads (Sample 2). A. Discussion Threads There were altogether 99 student messages (S-posts) and 17 facilitator messages (F-posts) in the selection of eight discussions. This represented more than 10% of the total number of messages in the discussion threads having at least five messages and dealing with student inquiries on the online courses. As reported in Table 1, the courses had different types of subject content. TABLE I NUMBER OF MESSAGES (POSTS) IN THE SAMPLES Course Course description Sample 1 Sample 2 S -p o st s F -p o st s S -p o st s F -p o st s C1 A theoretical subject with many concepts, definitions and descriptions. 16 0 9 1 C2 Composed of theoretical and practical subject content. Included subject content related to mathematics and digital logic. 14 2 10 1 C3 Covered more practical subject contents than theoretical contents. The students were expected to use open office applications. 14 3 20 6 C4 Composed of theoretical and practical subject content. The course is about the Internet and world wide web. 7 2 9 2 B. Coders and Coding Process We realised that our students had attempted to form their messages in English while thinking in Sinhala. As a result, they had used phrases that could not be understood by a coder who did not know Sinhala well. Therefore, we selected the coders from Sri Lanka. Both of them were university teachers of information technology in the country. One was in the research team and the other was an external researcher who participated only in the evaluation. The coders were instructed to follow the same coding procedure that Shea et al. [17] applied in a study to re-examine the CoI model. Additionally, during the discussion for negotiation, when there were disagreements between the two coders, the due reasons for the disagreement were enquired and noted down. Also, the coders‘ comments and suggestions to improve the coding schemes were gathered. C. Unit of Analysis Garrison, Anderson and Archer [22] stated that message- level unit was the most appropriate unit of analysis for identifying cognitive presence in their discussions. As messages are demarcated clearly in a discussion, it is easy to consider messages as the unit of analysis if students have posted only one idea in one message. However, in our case, students formed messages including more than one idea in one message. Also, they had not structured the messages into paragraphs, having one idea in one paragraph. Therefore, we decided to use the chunk of a message as the unit of the analysis. A chunk could be a complete message or a meaningful segment of a message, with a cue of a presence that is described in the CoI model. D. Inter-rater reliability measurements The coding decisions of the two coders were evaluated for inter-rater reliability using Cohen‘s Kappa (K) and Holsti‘s coefficient of reliability (CR). The reason for applying two reliability measurements: K and CR, was to eliminate the weaknesses associated with individual reliability > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 4 measurements and, thereby, to increase the validity of the results. Cohen kappa values were calculated using the equation on [26, p. 155]. Holsti‘s coefficient of reliability was calculated referring to [27, p. 140]. In reference to prevailing research, Rourke, Anderson, Garrison and Archer [28] report that inter-rater reliability should be more than 0.8000 for RC and more than 0.7500 for K in order to consider it as a very good agreement. The results of the present evaluation are discussed with respect to this satisfactory level of agreement. IV. RESULTS The evaluation of the existing CoI model with its three components: social, cognitive and teaching presence and the coding scheme prepared based on the analytical model of metacognitive presence did not result in initial IRR values at satisfactory levels. Especially the cognitive and metacognitive presence coding schemes were relatively less reliable than the two other coding schemes. The initial IRR values of the social presence coding scheme ranged from RC=0.5333- 0.8354 and K=0.3182- 0.6393 while that of the teaching presence coding scheme ranged from RC= 0.2857-0.8000 and K= -0.1667- 0.5714. Also, a considerable number of negotiated IRR values were below RC=0.8000 and K= 0.7500. For instance, the negotiated IRR of the cognitive presence coding scheme ranged from RC= 0.6522-0.7957 and K= 0.5014-0.6513. This signified that there was a poor agreement (according to [29]) between the two coders. Furthermore, the comments of the coders revealed that they had encountered difficulties in following the coding schemes. A. Difficulties and Suggested Modifications In order to make the coding schemes valid and highly reliable the schemes should be well comprehensible and easy to use. This can be achieved by adding more meaning and clarity to the descriptions of categories and indicators [10]. Doing this should assure that components of the model (presences) are properly described by the categories and indicators are with sufficient details to ―reflect the essence of the categories‖. [20, p. 68]. With this insight and considering the difficulties faced by the coders and their suggestions to improve the coding schemes, necessary modifications were introduced to the analytical model. The modifications made to the components of the model and their rationales are discussed below. Social presence coding scheme During the first evaluation, the coders experienced a difficulty in distinguishing conventional expressions from unconventional ones. Also, there were indications of students expressing emotions and tone of voice by using big letters, capital letters or colour text. Therefore, the two indicators, conventional and unconventional expressions were combined and the definition of the combined-indicator was modified accordingly, including other signs of affective expressions. The coders had long discussions that ended up with disagreements. For instance, at one occasion the coders did not come to an agreement due to ambiguity of the two indicators: ‗expressing emotions‘ in the affective category and ‗expressing appreciation‘ in the open communication category in the social presence coding scheme. The coders commented that it was not easy to determine whether an expression was not emotional when expressing an appreciation. For instance, they found chunks such as ‗Wow!‘ and ‗Best!‘, which could be interpreted both as appreciations and as emotions. Therefore, we decided to modify the indicator under open communication to ‗encouraging or complementing‘ The coders revealed that they were not sure of what ‗expressing values‘ exactly meant. Shea et al. [17] had also noted that the highly subjective nature of ‗expressing values‘ caused problems to the reliability of the social presence coding scheme. Therefore, we decided to clarify what ‗values‘ specifically means. Further, the comments of the coders suggested that ‗expressing values‘ could also be considered in the definition of the ‗self-disclosure‘ indicator. Therefore, we included ‗expressing values‘ in the ‗self-disclosure‘ indicator and modified the definition of ‗self-disclosure‘ to consider personal values such as beliefs, vision, attitudes and interests. There were a considerable number of clues that urged us to create two other indicators for the open communication category: ‗accepting mistakes‘ for the recognition of each other‘s contribution and ‗expressing willingness to support‘. Moreover, we modified the indicator ‗asking questions‘ to ‗requesting support‘, because we determined that it would be more suitable to identify students‘ requests for clarifications and information, which might not be in the form of a question. The examples provided for each category indicator of the social presence coding scheme were not helpful to our team. Therefore, we created examples relevant to our context and added them to clarify each category indicator. Cognitive presence coding scheme During the evaluation, the coders found it very difficult to distinguish the chunks to be matched with the ‗exploration‘ and ‗integration‘ categories in the tool. They commented on the ambiguity of the information in the coding scheme and suggested that the socio-cognitive processes and the examples provided in the coding scheme should be improved. Also, the coders noted that the instructions given in the coding scheme were not clear; the instructions implied that some chunks could be matched both with the ‗integration‘ and ‗exploration‘ categories. These comments and issues that the coders encountered were considered in improving the coding scheme and the modifications were done by referring to the explanations available in [4], [30] and [14]. The coders encountered messages containing expressions of satisfaction after solving the problem which caused the ‗triggering event‘. These messages could be interpreted as clues of resolutions. Therefore, the coders suggested us to add another indicator – ‗judging or evaluating and expressing satisfaction‘ to the ‗resolution or application‘ category. Additionally, the text in the examples column was replaced with a few sample codes that were more familiar to our context > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 5 (see Appendix). Five questions or problems were added to exemplify plausible causes of triggering events. Probable replies to these questions were added as examples for exploration, integration and resolution. Our intention was to make the analytical tool more easily understood by the coders. Also, we removed the extra instructions provided in the descriptor column to let the coders look freely for relevant clues and segment messages and to make coding decisions. Teaching presence coding scheme The teaching presence component of the model was evaluated by analysing both facilitator‘s and students‘ messages in four discussion threads. This signified the importance in changing the category indicators to make the tool more applicable to identify signs of teaching presence that emerged with the students‘ teaching activities during the discussions. The first category of the teaching presence construct is ‗designing and organisation‘ (DO). Shea et al. [17] had six indicators for this category. The first two indicators were ‗setting curriculum and communicating assessment methods to be used in the course‘ and ‗designing methods‘. The last was ‗macro-level comments about the course‘, which was defined as ‗provides rationale for assignment/topic‘. These indicators were found irrelevant to our context, because in our courses, the facilitator or the students were not supposed to set the course curriculum and assignments, design learning activities or provide macro-level instructions to do the course activities. These kinds of course planning and designing activities had been carried out by the instructional designers before starting the courses. However, the facilitator could use the discussion forums to post announcements relevant to the subject under discussion. Therefore, we added a new indicator, ‗informing notices‘, to the DO category. Based on the comments of the coders, the two indicators in the ‗facilitating discourse‘ category were slightly modified to make them more suitable to the context. The modified indicators were ‗encouraging, acknowledging or reinforcing student contribution‘ and ‗drawing in participants and prompting discussions‘ (FD-d). The discussions in the environment did not have any time limitations or restrictions. Therefore, the second indicator, FD-d, could also be considered as encouraging student participation. For this reason, in order to make the two indicators clearly comprehensible and applicable to the environment, we slightly modified the indicators to ‗acknowledging or reinforcing student contribution‘ and ‗encouraging or motivating students‘. The list of indicators in the direct instruction (DI) category was modified by adding two new indicators: ‗providing specific instructions‘ and ‗encouraging doing activities‘. These were determined during the first evaluation where the coders noticed that the students had provided task-specific instructions to peers and encouraged them to try out challenging activities. Moreover, based on the finding of the evaluation, two other indicators – ‗offering useful illustrations‘ and ‗making explicit references‘ – were modified by including ‗or examples‘ and ‗providing extra learning resources‘ respectively. The existing model had another indicator, ‗supplying clarifying information‘, which was quite similar to the two other indicators that were modified. However, the definition of the indicator described the teaching role of providing additional explanations. Therefore, in order to reduce the ambiguity of the indicator and make it more meaningful the existing indicator was replaced with ‗providing additional explanations‘. Moreover, the comments of the coders urge the importance of replacing the existing indicators of the assessment category with three new indicators. They are ‗judging, evaluating or checking the relevancy of message content‘, ‗assessing another student‘s knowledge‘ and ‗assessing an activity reported or presented in a message‘. Metacognitive presence coding scheme The coders suggested that we should add examples to clarify the meaning of the indicators in the coding scheme. They reported that it was difficult to understand the meaning of the term ‗knowledge of cognition‘ and to differentiate the meaning of ‗monitoring cognition‘ and ‗regulation of cognition‘. Therefore, in order to make the metacognitive presence coding scheme easily comprehended a set of examples were added to each category. Appropriate examples were selected from the sample coding in [16] and the rest were created. Furthermore, we removed some descriptive terms and added two indicators to the coding scheme. According to [31], knowledge of cognition (KC) incorporates general learning strategies and tasks that describe when, why or how one can perform an activity. In line with this reasoning, Akyol and Garrison reported that ―KC includes knowledge about cognition, cognitive strategies and tasks‖ [16, p. 184]. However, in order to describe category definitions of ‗monitoring cognition‘ and ‗regulation of cognition‘ (RC), they used the terms ‗task knowledge‘ and ‗strategies‘ respectively and that was incongruent with the description provided by Pintrich. Therefore, we removed the terms which created confusion and added one more indicator, ‗knowledge about general learning strategies and tasks‘, to the KC category. Also, we modified the list of indicators in the RC category by adding ‗suggesting taking an action‘ under the list of ‗applying strategies‘. This addition was made after noticing that our students had posted messages including statements such as ―Let‘s discuss about open office software‖ and ―It‘s better to give up‖. These statements could be interpreted as suggestions to take actions during the application of a strategy and thus considered as clues of RC. B. Results of the 2nd Evaluation The coding schemes with modifications were re-evaluated for their reliability. This second evaluation (Eval 2) was conducted two months after the first evaluation (Eval 1). The cognitive presence component of the CoI framework was explained to the coders referring to [4]. The same set of discussion threads (Sample 1) was given to the coders and the > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 6 same procedure was applied during the analysis. Finally, a new set of discussions (Sample 2) was analysed to determine whether we can use the adapted analytical model reliably to analyse online discussions in our courses. The coders analysed 99 students‘ messages and 17 facilitator‘s messages. They identified different numbers of chunks and altogether there were 599 coding decisions that had to be checked or discussed for negotiation. There was a significant increase even in the initial IRR values of all the coding schemes (see Graph 1 and 2). The negotiated IRR values of the schemes ranged from 0.9000 to 1.0000. Graph 1: IRR in RC measurement Graph 2: IRR in K measurement The reliability values resulted at the evaluation conducted with a new set of discussion threads (Sample 2) are shown in Table 2. TABLE 2 IRR VALUES OF THE EVALUATION WITH SAMPLE 2 Coding Scheme Course Initial IRR Negotiated IRR RC K RC K Social presence C1 1.0000 1.0000 1.0000 1.0000 C2 0.8814 0.6714 1.0000 1.0000 C3 0.8837 0.7283 1.0000 1.0000 C4 0.8889 0.7101 0.9744 0.9242 Teaching presence C1 0.9091 0.7500 1.0000 1.0000 C2 0.8750 0.8706 1.0000 1.0000 C3 0.9600 0.8870 1.0000 1.0000 C4 0.8750 0.6842 1.0000 1.0000 Cognitive presence C1 0.8235 0.6471 1.0000 1.0000 C2 0.5000 0.3402 1.0000 1.0000 C3 0.8679 0.7707 0.9600 0.9356 C4 0.7742 0.6583 1.0000 1.0000 Metacognitiv e presence C1 0.7826 0.5702 0.9600 0.8818 C2 0.8182 0.5560 0.9697 0.9040 C3 0.7179 0.4413 1.0000 1.0000 C4 0.9143 0.8182 1.0000 1.0000 Most of the IRR values of the evaluation conducted with Sample 2, reached levels of agreement which were more than RC=0.8000 = and K=0.5000. The negotiated IRR values ranged from RC=0.8600-1.0000 and K=0.8818-1.0000 (see Table 2). These can be interpreted as very good agreements between the two coders. Therefore, we can assume that the modifications we made could improve the schemes. Moreover, all the coding schemes seemed well applicable in our context. Each coding scheme supported to identify considerable numbers of clues. However - though it was encountered at very few instances- there were coder disagreements regarding decisions related to the analysis using social, cognitive and metacognitive presence coding schemes. At one instance, the coders had a disagreement regarding a decision pertaining to the ‗open communication‘ and the ‗affective‘ categories of the social presence coding scheme. The reason for this discrepancy was the ambiguity of the ‗expressing emotions‘ indicator in the affective category and the ‗expressing appreciation‘ indicator in the ‗open communication‘ category. The disagreement related to the cognitive presence coding scheme was due to un-clarity of a message posted by a student. The two coders interpreted it in two different ways that led one coder to match the whole message with the ‗exploration‘ category while the other coder matched it with the ‗integration‘ category. Each category of the metacognitive presence coding scheme could capture more than 29% of clues - out of the total number of metacognitive presences that could be identified. However, there were 4% of disagreements. One coder explains that those chunks could be considered as signs of bringing previously acquired knowledge to the discussion and thus they can be matched with the ‗knowledge of cognition‘ category. However, the other coder disagreed with the decision saying that there was not enough information to consider those chunks having signs of previously acquired knowledge. C. Relationship between metacognitive presence construct and other components The metacognitive presence coding scheme captured a considerable number of the chunks that could also be captured either by the teaching presence component or by the cognitive presence component of the tool. Moreover, we found that some of the chunks that had been identified using the cognitive presence or the teaching presence coding schemes were segmented further in matching with the categories in the metacognitive presence coding scheme. Also, there were > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 7 chunks that did not necessarily belong to the chunks with clues of cognitive presence or teaching presence, but had signs of metacognitive presence. For instance, in one discussion, the students considered what a closed system and an open system could be and tried to determine whether a prison could be a closed system or an open system. One student, who was found to have integrated his idea with others‘ opinions, added: ―Am I correct? ...prison: open prison, closed prison. You can’t explain these two types of prison only in a closed system.  I think this question makes some kind of confusion when thinking too much. It’s better to give up.‖ The last part of this message: ―I think this question...‖ could be treated separately from the rest of the message. During the analysis using the cognitive presence coding scheme, one coder suggested that it might be considered as a resolution. She reasoned that it might be interpreted as an attempt to seek a resolution or to end the discussion. The other coder did not agree and both decided to ignore that line in the message. However, when using the metacognitive presence coding scheme, both coders divided this line into two chunks and matched the first, ―I think this...too much‖ with the ‗monitoring cognition‘ category and the second, ―It’s better to give up‖ with the ‗regulation cognition‘ category. In another instance, a student provided a detailed description of binary arithmetic and reported: ―...If you feel anything not clear here please contact me. It is a pleasure to help you! I too had to struggle for days to have a good understanding. So, never give up! Wish you good luck!‖ The coders matched the chunk ―I too had to struggle for days to have a good understanding!” with categories in the metacognitive and social presence coding schemes, but not with any of the categories in the teaching or cognitive presence schemes. D. Typical issues and guidelines Methodological issues in online discussion content analysis have frequently been discussed in the literature (e.g. [32], [10] and [14]). In such articles, suggestions and advice that analysts can adhere to in order to handle issues related to reliability have been proposed. For instance by incorporating multiple coders, and using Cohen‘s Kappa to compensate for chance agreement and triangulation methods to increase the validity of the results. However, there is still a need for easily applicable instructions that can support novice analysts to achieve valid results. During our analysis, the main problem that we encountered was associated with the issue of culture. The students seemed to have formed messages in English while thinking in their mother tongue. Therefore, the coders had to pay extra attention to interpreting messages in the discussion threads. Further, the comments of the coders and our experience emphasised the importance of taking necessary steps to reduce the difficulty in analysing online discussion content in general. This resulted in the formulation of guidelines that could be followed in analysing online discussions. These guidelines are discussed below. Reformulate messages where it is essential In some cases we had to reform the messages to make them more comprehensible. Therefore, we suggest that the analyst of discussion threads should read all the discussion threads and try to understand the discussion before starting the analysis. While doing this, the analyst can carefully improve the clarity of the messages where it is essential. Study the context of the discussion In order to understand a discussion, the analyst may need to know the information related to the context of the discussion, which can be obtained from the online course environment where the discussion emerges. This contextual information will probably be essential to make decisions during the coding process. Therefore, if discussions are in printed form then the analyst should go back to the online learning environment and study the contexts of each discussion before starting the analysis. Understand the inquiry process The reason for using the CoI model was to understand the student inquiry processes that emerged for the purpose of solving problems related to subjects covered in the online courses. This understanding was mainly connected to the analysis conducted using the cognitive presence coding scheme. Therefore, analysts who wish to use the analytical tool of the CoI framework should have a thorough understanding of the CoI framework and the inquiry process. Comprehend the coding schemes Analysts should be able to comprehend not only the inquiry process and the CoI framework, but also the category definitions and indicators in each coding scheme of the CoI model. This will aid the analysts to investigating the chunks more precisely and, as a result, increase the reliability of the instrument. When there is more than one coder working with the analysis, they should grasp the instructions and information in the coding schemes together and build up a mutual understanding of the coding schemes. Consider only one coding scheme at a time The CoI model has four components: social, cognitive, teaching and metacognitive presence. Each of these components has three or four categories and altogether there are fourteen categories. Hence, it is not easy for a coder to either remember all the categories and their definitions or to refer to the schemes back and forth during the analysis. Therefore, analysts who are interested in investigating all the elements covered in the CoI model should use only one coding scheme at a time until all the discussions are analysed. This process should be repeated with all the coding schemes. Double check the work The coders who participated in our evaluation missed a considerable number of clues that could be identified by the model. This emphasises the importance of rechecking the analysis with a coding scheme before going to work with the > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 8 next scheme. V. DISCUSSION The evaluation reported on in this paper aimed to determine whether the CoI model could be used to analyse online discussion content in a learning environment prepared for distance learners in an Asian country. The evaluation was conducted with a sample set of discussion threads in four online courses that covered subject content relating to information technology. Out of these courses, three (i.e. Course 2, 3 and 4) had practical as well as theoretical subject content, while the other course did not have any laboratory- based learning activities. A. Reliability of the analytical tool The model was adapted considering our experience and the suggestions brought out by cotemporary researchers for better reliability of the model. For instance, Rourke, Anderson, Garrison, & Archer [33], who developed the social presence analytical tool, examined it by analysing discussions in two graduate-level courses; one in workplace learning and the other in distance learning. They reported that there were issues in investigating clues of expressions of emotions and humour. Shea et al. [17] used an adapted version of the CoI model and re-examined the model by analysing discussion content from two courses in business management at a State college in the United States. They experienced problems with the indicators ‗expressions of values‘ and other indicators that Rourke et al. [33] also confounded in the social presence coding scheme. Furthermore, we solved another issue encountered by our coders due to the ambiguity of the two indicators – ‗expressing emotions‘ in the affective category and ‗expressing appreciation‘ in the open communication category. After doing necessary modification and re-evaluating each construct of the analytical model, we could ensure that the constructs were more reliable than before. The reliability values of the adapted model were at higher levels than the reliability values found by Shea et al. [17]. Therefore, we believe that the CoI model with our modifications can be used reliably to analyse discussions in the online courses. More specifically, the results of the current study imply that the modifications made to the coding schemes are appropriate and relevant for conducting discussion content analysis in our online courses. Further research is welcome to affirm that the adapted model is more appropriate and reliable in other disciplines as well. B. Theoretical framework and metacognitive presence The theoretical framework of the CoI analytical model is composed of three major components – social presence, cognitive presence and teaching presence. Akyol and Garrison [16] introduced a new component, ‗metacognitive presence‘ to the theoretical framework and they placed it at the intersection of the cognitive and teaching presence components of the framework. However, in our evaluation, we found cues of metacognitive presence which could be matched neither with cognitive presence nor with teaching presence. This implies metacognitive presence does not fall only on the intersection of teaching presence and cognitive presence. It goes beyond the boundaries of cognitive and teaching presence. Therefore, the findings of the present study imply that the CoI framework may need further improvements to properly enclose the metacognitive presence construct. VI. CONCLUSION The present study was conducted to evaluate the CoI analytical model and the findings affirm that the model with our modifications was reliable and more applicable to analyse online discussions in our context. Based on the experience of our coders, a set of guidelines was formulated to lessen the difficulties pertaining to online discussion content analysis in general. Consequently, the validity of the results that analysts can find with this improved model and by following the set of guides will get increased. Therefore, this study adds more value to the CoI model and suggests that researchers will be able to use this model as a highly useful and reliable analytical tool. Hopefully, the implications from findings of the future research using this model will bring up more practical and fruitful suggestions to enhance students‘ learning experience in online learning environments. The coding process that we employed in the present study was very time consuming. This signifies the requirement of future research to develop application software to automate the analytical process and thereby evaluate students‘ learning in online discussions. For instance, the improved CoI model can be used to develop an application to automatically rate students‘ messages as soon as they are posted to online discussions. This will help teachers to more efficiently evaluate student activities in online discussions and consequently, students will get motivated to participate in online discussions and actively engage in the inquiry-based learning. Moreover, the present study encourages future research work to investigate possible improvements that can be done to the theoretical framework of the CoI model and properly enclose metacognitive presence construct which is useful for declaring information related to distance online learning environments. ACKNOWLEDGEMENT The study is financially supported by the Swedish International Development Cooperation Agency through the National e-Learning Centre Project at the University of Colombo, Sri Lanka. Our thanks should go to Mrs. T. Karunaratna, who participated in the evaluation as the external coder, and to those who provided feedback to improve the coding schemes. REFERENCES [1] Henri, F. (1992). Collaborative learning through computer conferencing: The Najaden Papers. (Ed. A. R. Kaye) pp. 117-136. Springer-Verlag, Berlin. > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 9 [2] McDonald, J. (1998). Interpersonal group dynamics and development in computer conferencing: The rest of the story. In Proceedings of 14th Annual Conference on Distance Teaching and Learning (pp. 243-48). Madison, WI: University of Wisconsin-Madison [ERIC Document ED422864] [3] Newman, G., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Interpersonal Computing and Technology, 3(2): 56-77. Available: http://qcite.qub.ac.uk/handle/123456789/5895 [4] Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3): 87-105. [5] Fahy, P. J., Crawford, G., Ally, M., Cookson, P., Keller, V., & Prosser, F. (2000). The development and testing of a tool for analysis of computer-mediated conferencing transcripts. The Alberta Journal of Educational Research, 66(1): 85–88. [6] McDonald, J., & Gibson, C. C. (1998). Interpersonal dynamics and group development in computer conferencing. The American Journal of Distance Education, 12(1): 7-25. [7] De Wever, B., Schellens, T., Valcke, M., & Van Keer, H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers and Education, 46(1): 6-28. [8] Krippendorff, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage. [9] Lombard, M., Snyder-Duch, J., & Bracken, C.C. (2004). Practical resources for assessing and reporting intercoder reliability in content analysis research projects. Available: http://www.temple.edu/mmc/reliability. [10] Garrison, D. R., Cleveland-Innes, M., Koole, M., & Kappelman, J. (2006). Revisiting methodological issues in the analysis of transcripts: Negotiated coding and reliability. The Internet and Higher Education, 9: 1-8. [11] Arbaugh, J.B., Bangert, A., & Cleveland-Innes, M. (2010). Subject matter effects and the community of inquiry (CoI) framework: An exploratory study. The Internet and Higher Education, 13(1-2): 37-44. [12] Swan, K., & Ice, P. (2010). The community of inquiry framework ten years later: Introduction to the special issue. The Internet and Higher Education, 13(1-2): 1-4. [13] Buraphadeja, V., & Dawson K. (2008). Content analysis in computer- mediated communication: analyzing models for assessing critical thinking through the lens of social constructivism. American Journal of Distance Education, 22(3): 130-145. [14] Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. The Internet and Higher Education, 13(1-2): 5-9. [15] Garrison, D. R., & Arbaugh, J.B. (2007). Researching the community of inquiry framework: Review, issues, and future directions. The Internet and Higher Education, 10(3): 157-172. [16] Akyol, Z., & Garrison, D.R. (2011). Assessing metacognition in an online community of inquiry. The Internet and Higher Education, 14(3): 183-190. [17] Shea, P., Hayes, S., Vickers, J., Gozza-Cohen, M., Uzuner, S., Mehta, R., & Rangan, P. (2010). A re-examination of the community of inquiry framework: Social network and content analysis. The Internet and Higher Education, 13(1-2): 10-21. [18] Puzziferro, M. (2008). Online technologies self-efficacy and self- regulated learning as predictors of final grade and satisfaction in college-level online courses. The American Journal of Distance Education, 22: 72–89. [19] McKerlich, R., & Anderson, T. (2007). Community of inquiry and learning in immersive environments. Journal of Asynchronous Learning Networks, 11(4). Available: http://www.incubatorisland.com/HTMLobj- 215/CoI_in_Immersive_Environments.pdf. [20] Garrison, D. R. (2007). Online community of inquiry review: Social, cognitive, and teaching presence issues. Journal of Asynchronous Learning Networks, 11(1): 61-72. [21] Anderson, T., Rourke, L., Garrison, D.R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2): 1-17. [22] Shea, P., & Bidjerano, T. (2010). Learning presence: Towards a theory of self-efficacy, self-regulation, and the development of a communities of inquiry in online and blended learning environments. Computers & Education, 55(4): 1721-1731. [23] Garrison, D. R., Anderson, T., & Archer, W. (2001b). Critical thinking and computer conferencing: A model and tool to assess cognitive presence. Available: http://auspace.athabascau.ca:8080/dspace/bitstream/2149/740/1/critical _thinking_and_computer.pdf [24] Pintrich, P.R., Wolters, C., & Baxter, G. (2000). Issues in the measurement of metacognition. (Eds. G. Schraw & J. Impara) pp. 43- 97. Lincoln, NE: Buros Institute of Mental Measurements. [25] Larkin, S. (2009). Socially mediated metacognition and learning to write. Thinking Skills and Creativity, 4(3): 149−159. [26] Neuendorf, K.A. (2002). The content analysis guidebook. California: Sage publications. [27] Holsti, O. R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison-Wesley Publishing Company. [28] Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12: 8-22. Available: http://hal.archives- ouvertes.fr/docs/00/19/73/19/PDF/rourke01.pdf [29] Banerjee, M., Capozzoli, M., McSweeney, L., & Sinha, D. (1999). Beyond kappa: A review of inter-rater agreement measures. Canadian Journal of Statistics, 27(1): 3–23. [30] Garrison, D. R., Anderson, T., & Archer, W. (2001a). Critical thinking, cognitive presence and computer conferencing in distance education. American Journal of Distance Education, 15(1): 7-23. [31] Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41(4): 219–225. Available: http://www.each.usp.br/cmapping/pdf/edmtxta10.pdf [32] Rourke, L., & Anderson, T. (2004). Validity in quantitative content analysis. Educational Technology Research and Development, 52(1): 5- 18. [33] Rourke, L., Anderson, T., Garrison, R., & Archer, W. (1999). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14(2): 50-71. Available: http://auspace.athabascau.ca:8080/dspace/handle/2149/732 http://qcite.qub.ac.uk/handle/123456789/5895 http://www.temple.edu/mmc/reliability http://www.incubatorisland.com/HTMLobj-215/CoI_in_Immersive_Environments.pdf http://www.incubatorisland.com/HTMLobj-215/CoI_in_Immersive_Environments.pdf http://auspace.athabascau.ca:8080/dspace/bitstream/2149/740/1/critical_thinking_and_computer.pdf http://auspace.athabascau.ca:8080/dspace/bitstream/2149/740/1/critical_thinking_and_computer.pdf http://hal.archives-ouvertes.fr/docs/00/19/73/19/PDF/rourke01.pdf http://hal.archives-ouvertes.fr/docs/00/19/73/19/PDF/rourke01.pdf http://www.each.usp.br/cmapping/pdf/edmtxta10.pdf http://auspace.athabascau.ca:8080/dspace/handle/2149/732 > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 10 APPENDIX Table 1 Coding scheme for social presence Category Code Indicators Definition Examples A ff e c ti v e e x p re s s io n (S -A E ) S- AE1 Using conventional or unconventional expressions to express emotions Mood or emotions expressed through symbols or words. Expressing emotions using repetitious punctuation, conspicuous capitalisation or emoticons. (May include big letters and text in different colours or fonts along with the normal text to express the tone of voice.) “...I‟m so ...” “Oh! Shit, I ...” “That‟s not what I mean.” “I just can't stand it when…!!!;” “ANYBODY OUT THERE!” “What does it mean!?!?” S- AE2 Using expressions of humour Using conventional strategies such as humorous banter, teasing and joking. “He heee..., no need to think, what the book says is correct and I‟m wrong.” S- AE3 Self-disclosure Disclosing details of personal life, expressing vulnerability or sharing personal beliefs, vision, attitudes and interests. “Where I work this is what we do...” “I just don‟t understand this question.” “I believe that we have right to see the assignment marks.” O p e n c o m m u n ic a ti o n (S -O C ) S- OC1 Continuing a thread Using reply feature of software, rather than starting a new thread. S- OC2 Quoting from others‟ messages Quoting complete or a part of a message posted by another. “What do you mean by „excess-k representation‟?” S- OC3 Referring explicitly to other‟s messages Directing references to contents of other‟s posts. “In your message, you talked about Excel not Calc.” S- OC4 Requesting support Requesting support or information from the students and the facilitator. (Includes questions.) “Could you please help me in ...?” “Please upload that file or send me the link. I‟d like to read it.” S- OC5 Encouraging or complementing Encouraging others or complementing others‟ or the content of their messages. “I really like your interpretation of the reading.” “Don‟t worry. You will find it easy.” S- OC6 Expressing agreement or disagreement Agreeing or disagreeing with others or content of their messages. “I was thinking the same thing. You really hit the nail on the head.” S- OC7 Accepting mistakes Acknowledging one‟s own mistakes and expressing gratitude for being made aware of them (recognition of each other‟s contribution). “Sorry, it‟s a typing mistake.” “Oh! I have made a big mistake. Thanks for showing it.” S- OC8 Expressing willingness to support Expressing willingness to support others whether with or without request for such help. “Contact me if you need further clarification.” “I can help you in Java and PHP.” S- OC9 Offering personal advice Offering specific advice to other students. “I recommend ADSL connection of .... The best thing to do first is...” G ro u p c o h e s io n (S -G C ) S- GC1 Vocatives Addressing or referring to the participants by names or as sister or brother. “I think John made a good point. Jessica, what do you think about it?” “No sister, that‟s not...” S- GC2 Addressing or referring to the group using inclusive pronouns Addressing the group as we, us, our group or as friends. “Our textbook refers to ...” “I think we veered off track...” S- GC3 Salutations and greetings Indicating purely social functions: greetings or closures. “Hi all” “Good luck!” S- GC4 Social sharing Sharing information unrelated to the course content but which helps maintain group cohesion. “We are having the most beautiful weather here. I will take some photos and send you soon.” Table 2 Coding scheme for metacognitive presence Categor y Code Description Indicators Examples > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 11 K n o w le d g e o f C o g n it io n M-KC (Entering Knowledge/M otivation) Entering knowledge and motivation associated with the inquiry process, academic discipline and expectancies. A more general aspect of metacognition observed anytime. Pre-Task Reflection Knowledge of the inquiry process Knowledge of critical thinking and problem solving Knowledge of factors that influence inquiry and thinking Knowledge of discipline Knowledge of previous experiences Knowledge of self as a learner Knowledge about general learning strategies and tasks Entering motivational state Expectancy of success “Based on a combination of my past reading and experience, I define ...” “I remember in my first year teaching online …. It highlighted for me the importance of engaging activities.” “We know how to work with Windows… We have experience...” “I am quite slow in reading.” “I make short notes while listening to help me understand.” “I am certain that I can understand even the most difficult text that they are going to discuss.” “We will be able to solve this…” M o n it o ri n g o f C o g n it io n M- MC (Assessment) Observed during the leaning process. Awareness and willingness to reflect upon the learning process. Assessment of task, understanding progression and effort required is an important monitoring function. Reflection on Action Declarative; judging Commenting on task, problem or discussion thread Asking questions to confirm understanding Commenting about self and others‟ understanding Making judgments about validity of content Commenting on or making judgments about the strategy applied Asking questions about progression or stalling Assessing motivational state and effort required “I have understood blended learning to be a …” “I like your eloquently worded definition…” “Good points.” “…. well today I have learned something about a …” “I think I have been able to think of an example for almost each of the models presented in …” “You make an interesting point when you observe …” “I am not certain why this is true a priori.” “Am I correct?” “You all have done very well…” “I am interested in reading from Tom's list.” R e g u la ti o n o f C o g n it io n M-RC (Planning) Observed during the leaning process. Enactment and control of the learning process through the employment of strategies to achieve meaningful learning outcomes. Reflection in Action Procedural; planning Setting goals Applying strategies o Providing/asking for support o Challenging self or others o Asking questions to deepen thinking o Asking for clarification o Request information o Self questioning o Suggesting taking an action Questioning progression, success Taking control of motivation and effort Facilitating/directing inquiry “Your thoughts?” “I think I need to see more supporting research for the idea that …” “One of your solution is …Would it be feasible within …” “Also, I think I am going to need help in understanding” “I am just curious about the social processes … and how they might help learning” “Shall we give up this now and go to the next topic?” “Will we be able to finish this on time?” “Let‟s use the new software and share our experience.” > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 12 Table 3 Coding scheme for teaching presence Categor y Code Indicators Definition Examples D e s ig n a n d o rg a n is a ti o n (T -D O ) T- OD1 Informing notices Provides information related to changes or updates to the course content or any other notices specific to the subject under discussion “A new version of this lesson is uploaded to the course. Please have a look at it before answering to this activity.” “You can access the lessons from here...” T- OD2 Establishing time parameters Communicate important due dates/time frames for learning activities to help students to keep pace with the course. Please post a message by Friday T- OD3 Utilising medium effectively Helps students to find appropriate places to discuss concerns. Therefore, attempts to organise/manage the discussions properly. Assists students to use LMS features for learning activities and resolving technical problems. “This has been discussed in the forum... Please post your question there.” “Try to address issues that others have raised when you post.” T- OD4 Establishing netiquette Helps students understand and practice the kinds of behaviours that are acceptable in online learning, e.g., providing documentation on polite forms of online interaction. “Keep your messages short.” “Remember, all uppercase letters is the equivalent of “shouting.” F a c il it a ti n g d is c o u rs e (T -F D ) T-FD1 Identifying areas of agreement/disagree ment Assists in identifying areas of agreement and disagreement on course topics in order to enhance student learning. “Joe, Mary has provided a compelling counter-example to your hypothesis. Would you care to respond?” T-FD2 Seeking to reach consensus/understa nding Assists in guiding class towards agreement about discussion topics in a way that enhances student learning. “I think Joe and Mary are saying essentially the same thing.” T-FD3 Acknowledging or reinforcing student contributions Acknowledges student participation in the course, e.g., reply in a positive and encouraging manner to student submissions. “Thanks, for your post Alex.” “Thanks for your contribution.” T-FD4 Encouraging or motivating students to participate in the discussion Assists students engaging and participating in productive dialogue. “Who else can provide an answer to what Peter asked?” “Any thoughts on this issue?” T-FD5 Setting climate for learning Encourages students to explore concepts in the course, e.g., promotes the exploration of new ideas. “Don't feel self-conscious about „thinking out loud‟ on the forum. This is the place to try out ideas after all.” T-FD6 Re-focusing/re- addressing discussion on specific issues Helps focus discussion on relevant issues and keeps participants on topic. “I think that's a dead end. I would ask you to consider...” “Is that all what you have to say about this topic? What about …?” T-FD7 Summarising discussion Reviews and summarises discussion contributions to highlight key concepts and relationships to further facilitate discourse. “The original question was…. Joe said…Mary said…” “We concluded that…We still haven't addressed…..” D ir e c t In s tr u c ti o n (T -D I) T-DI1 Providing specific instructions or advices Provides task-oriented instructions or advice for learning. “Create a table by yourself and try to add multiple keys...” “You should read the question carefully...” T-DI2 Offering useful examples or illustrations Explains subject content using examples and illustrations (an attempt to make course content more comprehensible). “Look at the following figure. It shows you how to print...” “This example will help you to understand...Let‟s assume that ...” T-DI3 Providing additional explanations Attempts to reduce confusion or misconceptions about course content by providing additional explanations. “Let me provide you with some additional detail explaining how this device works.” T-DI4 Making explicit references or providing extra learning resources Provides useful and relevant information to find solutions or for further clarifications. “This will help you to clarify your doubts; http://….” T-DI5 Encouraging doing activities Encourages or motivates students to complete learning activities and try out challenging tasks that are required to be completed during the course. “Try the activity 5. It‟s very easy.” “You can do it if you follow ...” T-DI6 Responding to technical concerns Provides direct instructions on technical questions related to the online learning environment. “If you want to include a hyperlink in your message, you have to ...” A s s e s s m e n t (T -A S ) T-AS1 Providing constructive feedback to student posts Judges and evaluates the relevancy and usefulness of the information posted in messages and provides comments or tips on improvement. “Very good, Nadia. You have posted some very useful information.” “Are sure what you have posted here is correct. You better check it again John.” > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 13 Table 4 Coding scheme for cognitive presence Phas e Descriptor Code Indicators Socio-cognitive process Examples (T1,...,Tn are triggering events and T1,...,Tn are replies to the issues raised at the triggering eventsT1,..,Tn respectively.) T ri g g e ri n g e v e n t (C -T E ) Evocative Stimulate one‟s curiosity Core organising concept or problem that learners can relate to from their experience or previous studies Framing the issue and eliciting questions or problems that learners see or have experienced Assessing state of learners knowledge and generating unintended but constructive ideas C-TE1 Recognisi ng problem Presents background information that may culminate in a question or presents a problem/issue. T1. “In section 5, page 152 of the student manual says „solid states‟... Could you please explain what it means?” T2. “I think the statement „the Internet uses TCP standards in data transmission‟ is correct. But in a Quiz, it is considered as incorrect. Can it be a mistake? Please explain.” C-TE2 Sense of puzzlemen t Questions or messages that take the discussion in a new direction. T3. “Sometimes ago, I studied what „bit‟ and „byte‟ are. But now, I can‟t remember and I am confused. Can someone explain what they are?” T4. “I wanted to print cell borders of a Calc worksheet. But failed. Is there anybody who has done it before?” T5. “Are touch-screen laptops better than normal laptops?” E x p lo ra ti o n (C -E X ) Inquisitive Understand the nature of the problem and then search for relevant information and possible explanation Group activities- brainstorming Private activities- literature searches Manage and monitor this phase of divergent thinking in such a way that it begins to be more focused C-EX1 Exploratio n within the online community Unsubstantiated agreement or disagreement/contradiction of previous ideas. T2  “I don‟t agree. It is incorrect.” “I agree with you.” C-EX2 Exploratio n within a single message Many different ideas/themes presented in one message. T1  “Dictionary meaning of „solid state‟ is.... In a past exam paper I found it defined as „...‟ But I have been taught it as „...‟” C-EX3 Informatio n exchange Personal narratives or descriptions (not necessarily regarding personal experiences) or facts (i.e., from sources such websites, articles, programmes, etc.) Adds points but does not systematically defend/justify/develop addition. T4  “http://www..... This online video might help you to understand how to print cell borders.” C-EX4 Suggestio ns for considerati on Author explicitly characterises message as exploration T3 [After bringing out some information about bit and byte] “Does that seem about right?” “Am I way off the mark?” C-EX5 Leaping to conclusion Offers unsupported opinions T2  “...It‟s a mistake.” T4  “Cell borders of a worksheet cannot be print.” In te g ra ti o n ( C -I N ) Tentative Focused and structured phase of making meaning Decisions are made about integration of ideas C-IN1 Integration among group members Reference to previous message followed by substantiated agreement or disagreement. Building on, adding to others‟ ideas. T2  “I don‟t agree with you because...” “I agree because...” “According to what Renuka noted, ... But I think ...” C-IN2 Integration within a single message Justified, developed, defensible, yet tentative hypotheses. T4  “I used this free tutorial, http://.... It explains how to print worksheets with cell borders. According to that, first you have to ....” C-IN3 Connectin g ideas Integrating information from one or more sources – textbooks, articles, personal experience, other posts or peer contribution. T5  “As Neel said, now there are laptops with touch screens. See the attached picture. But there is a problem with these laptops. Read this, http://... Therefore, I think...” C-IN4 Creating solutions Explicit characterisation of message as a solution by participant. T4  “Here is the answer; you can print cell borders like this... Format>Page>Sheet tab>...” R e s o lu ti o n /a p p li c a ti o n (C -R A ) Committed Reducing complexity by constructing a meaningful framework or discovering a contextually specific solution Confirmation or testing phase may be accomplished by direct or vicarious action. Resolution of the dilemma or problem C-RA1 Vicarious application to real world testing solutions Providing examples of how problems were solved or evidences of successful application. T4  “How I printed a Calc worksheet with cell borders was...” “It did not work at first. But when I selected some lines of text and tried again then it worked...” C-RA2 Defending solutions Defending why a problem was solved in a specific manner. T4  “Here is the modified list of steps to print a worksheet with cell borders. I did a small change to the second step of Mahela‟s procedure. Because I could not open the Print dialog box by following it as it was.” C-RA3 Judging or evaluating and expressing satisfactio n Judgement or evaluation followed by an expression of satisfaction after solving the problem or issue that caused the triggering event. T1  “...I understood what „solid state‟ means. Thanks.” T2, T3, T5  “...Thanks for the explanation. I got my doubt cleared/ problem solved.” T4  “...Thanks a lot. I followed your instructions and printed a worksheet with cell borders.” > REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < December 2012 International Journal on Advances in ICT for Emerging Regions 14