Australasian Journal of Educational Technology, 2019, 35(5). 60 What do they really like? An investigation of students' perceptions of their coursework in a fully online educational technology program Jesus Trespalacios, Patrick R. Lowenthal Boise State University The higher education market space has become increasingly competitive during the past few years. For instance, there are now hundreds of fully online master’s programs in educational technology. Given this increased competition, it is more important than ever for those working in graduate education to understand what students, and specifically those who have graduated from their program, like and dislike about their coursework. We constructed a survey using Rovai’s Classroom Community Scale (CCS) and some additional questions about the easiest, most difficult, and favourite courses to investigate what graduates thought about the coursework of the program they just completed. Results suggest that students in the program prefer courses with practical content, continuous interactions, and hands-on projects; and they dislike courses that are too theoretical. We conclude with implications for practice and additional research. Introduction Distance education enrolments have increased over the past decade (Seaman, Allen, & Seaman, 2018). However, despite this growth, online courses are estimated to have higher attrition rates than traditional face-to-face courses (Angelino, Williams, & Natvig, 2007; Boston, Ice, & Gibson, 2011; Patterson & McFadden, 2009). Research suggests that one reason students drop out from online courses or programs is because of feelings of isolation and loneliness (Ludwig-Hardman & Dunlap, 2003; Richardson, Maeda, Lv, & Caskurlu, 2017). One way to help students persist and address feelings of isolation and loneliness is through establishing and maintaining social presence (Boston et al., 2009; Boston et al., 2011); that is, helping students to develop a sense that other people are there and real in the online classroom (Dunlap & Lowenthal, 2014; Lowenthal, 2010). Another way to address feelings of isolation and loneliness (and in turn possibly address student attrition) is through developing a sense of community in online courses (Garrett Dikkers, Whiteside, & Lewis, 2012; Picciano, 2002; Rovai, 2002a; Whiteside, 2015; Whiteside, Garrett Dikkers, & Lewis, 2014). However, developing a sense of community in formal online courses can be difficult (Wilson, Ludwig-Hardman, Thornam, & Dunlap, 2004). This is in part due to a few different factors outside of students’ control. Students find themselves in a situation where participation is required over a fixed length of time in an environment that they must intentionally log into. Addressing students’ feelings of isolation and loneliness and ultimately better understanding why students persist or drop out of online courses and programs is becoming increasingly important in today’s competitive higher education market space (Bawa, 2016; Park & Choi, 2009; Shea & Bidjerano, 2014; Yang, Baldwin, & Snelson, 2017). Students now have more choices than ever of where they can complete their post-secondary education. As a whole, colleges and universities want their students to complete the programs they begin, feel satisfied with their learning experiences, apply what they learned during their studies in the workplace, and tell others about the good educational experiences they had. We contend that online programs must be in a continuous state of program improvement to meet these goals. As evident in the previously mentioned attrition data alone, positive online learning experiences do not magically happen on their own or for all learners (cf. Vlachopoulos, 2016). Program evaluation is one essential component of program improvement, especially for online programs (Rovai, 2003; Rovai, Ponton, & Baker, 2008). Institutions use program evaluation to ensure that students are learning what they are supposed to be learning and that instructors are teaching what they are supposed to be teaching. In other words, institutions monitor their students’ experiences for both curriculum development and quality assurance purposes (Lawless & Richardson, 2004). Two of the more traditional metrics used during a program evaluation are student grades and end-of-course student evaluations. Grade inflation though, makes using grades problematic in many programs (Allen, 2005; Johnson, 2006), which leaves many relying heavily on end-of-course evaluations (Rogers, 2003; Vogt & Slish, 2011; Winters & Payne, 2005). While course evaluations are often criticised for not being valid measures of teaching Australasian Journal of Educational Technology, 2019, 35(5). 61 effectiveness, research does suggest that course evaluations are adequate measures of student satisfaction (Lowenthal, Bauer, & Chen, 2015) and research suggests that student satisfaction is important for administrators of online programs to investigate (Kuo, Walker, Belland, & Schroder, 2013). The problem with using course evaluations to evaluate an entire online program is that they are completed after each course is over and therefore are unable to provide a larger perspective of an online program. As such, it is becoming increasingly common for institutions to investigate students’ perceptions at the end of their programs through some type of exit survey (Lawless & Richardson, 2004; Tight, 2012; Young, 2006). This study attempted to contribute to the knowledge base by exploring graduates’ opinions about their satisfaction of an online graduate program. The purpose of this study was to investigate what recent graduates of a fully online educational technology graduate program thought of their online coursework. More specifically, we were interested in identifying which courses (both in terms of the content and the instructor/learning experience) students thought were the easiest and the hardest, which courses were their favourite, as well as which courses had the strongest sense of classroom community. We collected this data in an effort to identify what is working and what is not working well in this fully online master’s program. In this paper, we report the results of our inquiry and the implications for the research and practice of online learning. Background This study was framed by two foundational ideas. First, online students should be satisfied with their overall learning experience. If they are not satisfied, they are more likely to stop logging into virtual classroom environments (e.g., the learning management system) and enrolling in additional courses. Second, online students should ideally develop a sense classroom community in their online courses and program of study. We describe these ideas in more detail in the following sections. Students’ satisfaction Students’ satisfaction is an important program outcome (Kuo, Walker, Schroder, & Belland, 2013; Tight, 2012). Multiple studies have investigated factors that promote student satisfaction in online education (Table 1). The importance of student satisfaction is perhaps best highlighted by its inclusion in one of the leading quality assurance frameworks of online learning, the Online Learning Consortium’s (OLC) 5 Pillars of Quality Framework, which consist of: learning effectiveness, scale, access, faculty satisfaction, and student satisfaction (Baldwin, Ching, & Hsu, 2018; Moore, 2002). The OLC’s website currently describes the student satisfaction pillar as “students are pleased with their experiences in learning online, including interaction with instructors and peers, learning outcomes that match expectations, services, and orientation” (Sloan Consortium, 2018). Other researchers have focused on the importance of student satisfaction when evaluating online learning. For instance, McGorry (2003) argued that the following seven constructs are needed to evaluate quality and learning in online courses: flexibility, responsiveness and student support, student learning, interaction, technology and technical support, and student satisfaction. Rovai (2003) developed a framework for conducting evaluations of online programs that focused on student performance, program and cost- effectiveness, technology and support services, course design and instruction, and teacher and student satisfaction. Based on the previous research on students’ satisfaction, Sun, Tsai, Finger, Chen, and Yeh (2008) developed a framework where they identified 13 factors classified in 6 dimensions: (1) learner (attitude toward computers, computer anxiety, internet self-efficacy), (2) instructor (response timeliness and attitude toward e-Learning), (3) course (flexibility, quality), (4) technology (technology and internet quality), (5) design (usefulness and ease of use), and (6) environmental (diversity in assessment and learner perceived interaction with others). Guided by the framework of Sun et al. (2008), we were interested in what we could learn from our graduates about our program both in terms of course design (design dimension) as well as interaction strategies (instructor and environmental dimensions). For instance, rather than just relying on general best practices when it comes to the courses we design and teach - such as, establish social presence, create a community of learners, respond to emails within 24 hours - we wanted to have a better idea of the types of courses and learning experiences that graduates from our program valued the most and why they valued them. Australasian Journal of Educational Technology, 2019, 35(5). 62 Table 1 Factors promoting students’ satisfaction in online environments Studies Factors Bolliger and Martindale (2004) Instructor variables, technical issues, and interactivity Daugherty and Funke (1998) Meaningful learning, access to content knowledge, motivation, and convenience Eom, Wen, and Ashill (2006) Students’ self-motivation and learning style, instructors’ knowledge and feedback, interactions, and course structure Kuo, Walker, Schroder, and Belland (2013) Learner-instructor interaction, learner-content interaction, and Internet self-efficacy Li, Marsh, Rienties and Whitelock (2017) learning design (e.g., teaching materials, assessment strategies, workload) and long-term goals of learners (i.e., qualifications and relevance of modules for learners’ professional careers) Sun, Tsai, Finger, Chen, and Yeh (2008) Learner computer anxiety, instructor attitudes toward e-learning, course flexibility and quality, usefulness and ease of use of the e- learning system, and diversity in assessment Swan (2001) Clarity of design, interaction with instructors, and active discussion among course participants Sense of community During the 1980s and 1990s, educators increasingly focused on the importance of community in learning environments (Brown & Campione, 1994; Hill, 1985; Jonassen, 1995; Rogoff, 1994). In fact, educators keep talking about communities of learners (Bransford, Brown, & Cocking, 2000; Gannon‐Leary & Fontainha, 2010), communities of practice (Barab & Duffy, 2000; McDonald & Cater-Steel, 2017; Wenger, 1998, 2000), and professional learning communities (DuFour & Eaker, 1998; Stoll, Bolam, McMahon, Wallace, & Thomas, 2006). Community arguably takes on even more importance for online educators because online students typically do not meet and learn in the same place and time, which as mentioned earlier can lead to feelings of isolation and loneliness (Haythornthwaite, Kazmer, Robins, & Shoemaker, 2000; Ludwig-Hardman & Dunlap, 2003; Richardson et al., 2017) and high attrition rates (Angelino et al., 2007; Bawa, 2016). As such, researchers have been investigating the role of community in online courses for more than 15 years (Chang, 2012; Hill, 2012; Kear, 2011; Lally & Barrett, 1999; Luppicini, 2007; Palloff & Pratt, 2007; Sloman & Reynolds, 2003; Swan & Shea, 2005). During this time, researchers have consistently found a relationship between a sense of community and student satisfaction (Baturay, 2011; Brown, 2001; Liu, Magjuka, Bonk, & Lee, 2007; Ni & Aust, 2008; Ouzts, 2006; Shea 2006; Shea, Li, Swan, & Pickett, 2005); that is, past research suggests that students are more satisfied in online courses with a strong sense of classroom community. Researchers have also found a relationship between a sense of community and perceived learning. In other words, students often report that they learn more in online courses with a strong sense of classroom community (Baturay, 2011; Liu et al., 2007; Ni & Aust, 2008). Early online educators began investigating ways to develop community online. For instance, Swan and her colleagues published some of the earlier research on learning communities in online courses. Swan et al. (2000) found that: (1) consistent and transparent course design, (2) regular and constructive instructor interaction with students, and (3) active discussions, influenced the success of online courses. They went on to argue that these three factors in turn help lay the foundation for knowledge building communities. Later, Swan (2002) investigated course design features and student immediacy behaviours that influence the social development of learning communities. Swan reiterated the importance of clear course structure, interactive instructors, and dynamic discussions but she also argued for the relevance of students’ verbal immediacy behaviours - affective, cohesive, and interactive - in online discussions to develop a sense of community among classmates. Around the same time, Rovai (2001) examined course interactions, sense of Australasian Journal of Educational Technology, 2019, 35(5). 63 community, and learner feedback. He found a moderate relationship between classroom community and the number of times someone posted. Rovai (2001) also pointed out that an instructor’s writing styles, an instructor’s immediacy, course content, or length of course could also influence the development of community. Shea and his colleagues conducted a number of studies on learning communities and presence (see Shea, 2006; Shea et al., 2005; Swan & Shea, 2005). For instance, Shea et al. (2005) investigated the role of teaching presence in developing community in online courses. They found that teaching presence is related to classroom community; more specifically, they explained “that a strong and active presence on the part of the instructor - one in which she or he actively guides the discourse - is related to students’ sense of both connectedness and learning” (p. 71). In summary, multiple studies have highlighted the importance of student satisfaction and developing a sense of community within online courses and programs. Given this, we investigated what students like or dislike about specific courses and learning experiences (e.g., as is done with end-of-course questionnaire) or program of study (e.g., in terms of programs outcomes often listed in exit surveys). We also investigated what graduates of our program, that is, those who successfully completed our Master of Educational Technology program, remember liking or not liking about the courses they completed (e.g., in terms of community, workload, perceived learning). Thus, the research questions that guided this study were: 1. What were graduates favourite courses and why? 2. What were graduates most difficult courses and why? 3. What were graduates easiest courses and why? 4. What courses and instructors did graduates perceive had the strongest sense of community and why? Methodology Participants and context of the study This study was conducted at a metropolitan research university located in the Pacific Northwest of the United States. The university enrols more than 22,000 students in over 200 undergraduate and graduate programs. Approximately 60% of the students are enrolled full time. Enrolments in online courses continue to grow; currently 39% of students take at least one online course during their program of study. The department in which this study was conducted is one of the larger educational technology departments in the country with over 700 students in 2015, the time that this study was completed (For additional information about the educational technology program, see Appendix A). The department offers two master’s degrees, three graduate certificates, an educational specialist degree, and a doctorate in education, all completely online. Only about 15% of the department’s students live in the state in which the university is located; while the majority of students live in the United States, each year the department has students living outside of the United States. Currently about 60% of students are female and 40% male; and about 82% of students are white. The majority of students are educators or technology professionals with a bachelor’s degree, working in K-12 or higher education environments. An increasing number of graduates go on to teach or be involved with online learning in some way in their professional career. Currently, the department has 14 tenured or tenure-track faculty in different ranks and 16 adjunct faculty that normally teach two or three courses per semester. The authors of this manuscript are associate professors, with more than 5 years of experience teaching in the department. Both of them design and teach classes mostly related to instructional design and distance education. The Master of Educational Technology (MET) program, from which all of the participants of this study graduated, consists of five required core courses and a number of elective three-credit online courses that are dedicated to a particular subject. These courses are offered during the fall and spring semesters over 15 weeks; some courses are also offered in an accelerated format throughout the summer over 7 weeks, 10 weeks, or 14 weeks. Australasian Journal of Educational Technology, 2019, 35(5). 64 The five required core courses include: 1. Introduction to Educational Technology 2. Creating Educational Websites 3. Instructional Design 4. Theoretical Foundations of Educational Technology 5. Evaluation for Educational Technologists Each student selects 5 out of 28 possible courses to complete as their electives for the program; the electives are grouped into different areas of study (e.g., e-learning design, online teaching, emerging technologies, etc.). All of the MET courses are aligned to the Association for Educational Communications and Technology (AECT) standards, which consist of 5 standards that contain 25 performance indicators in total (Association for Educational Communications and Technology [AECT], 2012). All courses are capped at 23 students per section. Data collection and analysis Based on previous research as well as the desire to learn about our graduates’ experiences taking specific courses, a cross-sectional survey research design study was employed to answer the research questions (Creswell, 2015). The online survey, administered in Qualtrics, consisted of 34 questions; 28 questions were closed-ended and 6 were open-ended (Appendix B). The first set of questions asked students to identify the easiest, most difficult, and favourite courses they took during their program of study and to explain why they selected them. We then asked students to reflect on the course they completed with the strongest sense of community and then to identify the course and who taught it. The participants were then asked to complete Rovai’s (2002b) Classroom Community Scale (CCS) based on the course they identified with the strongest sense of community. The CCS consists of 20 Likert-scale questions composed of two subscales of connectedness and learning. As defined by Rovai (2002b): Connectedness represents the feelings of students regarding their cohesion, spirit, trust, and interdependence. Learning represents the feelings of students regarding the quality of their construction of understanding and the degree to which they share values and beliefs concerning the extent to which their learning goals and expectations are being satisfied. (p. 325) The CCS includes questions such as: “I feel that students in this course care about each other”, “I feel reluctant to speak openly”, and “I feel uncertain about others in this course”. Face validity and construct validity of the instrument have been established in previous studies (Rovai, 2002b; Rovai & Baker, 2005). Using a convenience sample strategy, researchers surveyed participants over two semesters in 2015. These two groups were selected because they were the most recent graduates at the time this research study was designed. An invitation to participate in the study was emailed to students in the MET program the week after they graduated from the program. Overall, 139 graduates were invited to participate in the study; 72 graduates ended up completing the anonymous survey and participating in the study voluntarily, for a 51.8% response rate. Data was downloaded from Qualtrics into a spreadsheet. We identified and calculated the courses with the most votes for each question and then used a constant comparative open-coding technique to analyse the open-ended responses. Codes were grouped together and themes identified. These different themes were then labeled and a percentage score representing the frequency at which they were selected was calculated. There were cases that units of information contained information for two categories in which case 1 point was given to both of them. The study was approved by the Institutional Review Board (IRB) at the university where the study was conducted. Results Favourite online course Four different courses emerged as students’ favourite course in the program: (a) Creating Educational Websites (26.4%), (b) Educational Games and Simulations (8.3%), (c) Mobile App Design (6.9%), and (d) Technology-supported Project-based Learning (6.9%) (Table 2). While there was not one single course Australasian Journal of Educational Technology, 2019, 35(5). 65 selected by all of the participants, more than a quarter of the respondents identified Creating Educational Websites as their favourite course. In Creating Educational Websites, students learn how to use HTML, CSS, and Adobe Dreamweaver to create educational websites. Creating Educational Websites is a core course that is often taken during the first semester of the MET program. There were two main reasons, according to the open-ended responses, why students identified Creating Educational Websites as their favourite course. The first reason according to the participants was the overall content of the course (n = 15; 79%). Learning how to create websites was a new skill for many students. For instance, one respondent said: “It was exciting learning how to code with HTML5 and CSS3 to build webpages in Dreamweaver. I would have never learned to do that without this class”. In addition, another student reported: “I learned a great deal of practical skills that directly impacted my curriculum - namely, using HTML and CSS to create a WebQuest. It was something I had wanted to learn for a long time but never had the means or time”. The second most commonly, but significantly less frequently, cited reason why students identified Creating Educational Websites as their favourite was because of the instructor who taught the course (n = 4; 21%). For example, one student indicated; “The instructor made it interesting. He was very knowledgeable and enthusiastic himself. He showed inquiry himself and was extremely supportive and helpful when students got stuck”. Another student commented; “I learned something totally new to me (HTML and creating Web pages for instruction). I was able to apply my new learnings and be creative as well. The instructor gave immediate and very detailed feedback that helped me a lot”. Unlike Creating Educational Websites, the other three courses selected as favourite courses were elective courses - that is, courses selected based on students’ personal interest. It is important to note that two out of three of these other courses identified by students as their favourite were comparatively highly technical in nature, like Creating Educational Websites. Table 2 The top five favourite courses in the online graduate program (number of responses=72) Top-rated courses Frequency Percentage Creating Educational Websites 19 26.4% Educational Games and Simulations 6 8.3% Mobile App Design for Teaching and Learning 5 6.9% Technology-Supported Project-Based Learning 5 6.9% Research in Educational Technology 4 5.6% YouTube for Educators 4 5.6% Advanced Online Teaching 4 5.6% Easiest online course We were also interested in identifying what courses graduates considered to be the easiest course(s) in the program. Introduction to Educational Technology was selected by more than a third of the graduates as the easiest course in the program. This is often the first course students take in the MET program, and it focuses on providing an overview of educational technology. Graduates who chose this course as the easiest (n = 25) stated that it was the course content that made this the easiest course in the program; the graduates reported that they were already familiar with much of the course content. Take for instance the following quote: “[T]his course was easy to me because I already knew most of the platforms we used and the learning curve wasn't as large for me”. Another student stated: “It was truly an introduction to the program. I learned new things, but it was not overwhelming. The instructor was very clear and the projects were not overly complicated”. Australasian Journal of Educational Technology, 2019, 35(5). 66 While more than a third of graduates identified Introduction to Educational Technology as the easiest course they took, around 41.6% of the graduates (30 out of 72) identified four other courses, thus suggesting that there still was not much agreement amongst respondents (Table 3). For instance, nine (12.5%) graduates picked Creating Educational Websites, nine (12.5%) graduates selected Graphic Design for Learning, and eight (11.1%) graduates picked Multimedia as the easiest courses in the program. When looking at the reasons why these other courses were selected, we found that students mostly pointed out that they had prior knowledge and experience in the subject of the course. However, a few also pointed out that the quality of instruction or how quickly they could complete the assignments made the courses the easiest. It is important to note that only two students identified the same course as the easiest and their favourite course. Table 3 The top five easiest courses in the online graduate program (number of responses=72) Top-rated courses Frequency Percentage Introduction to Educational Technology 25 34.7% Creating Educational Websites 9 12.5% Graphic Design for Learning 9 12.5% Multimedia 8 11.1% YouTube For Educators 4 5.6% Hardest online course We were also interested in what graduates remembered as the hardest course they took in their program. While 22 (31.4%) graduates reported that Theoretical Foundations of Educational Technology was the hardest course they took in the program, over a quarter, 18 (25.7%), of graduates also identified Evaluation for Educational Technologies as the hardest. These two courses (as well as Portfolio which was identified by 8 [11.4%] graduates) are different than most of the other courses in the program because they are less technology or tool-focused. The graduates described multiple reasons why Theoretical Foundations of Educational Technology was the hardest in the program. The most popular reason why this course was the hardest was because of the instructional activities (described by 35%) and the content of the course (described by 29%). The graduates specifically remembered the final writing project. One respondent wrote: “I would rather say ‘challenging’, and this although I am extremely familiar with educational theory. The challenge was writing the paper and assimilating a lot of research in order to formulate my own thesis. I did enjoy this course however!” Another stated: “I thought this was hard because it was a lot of deep thought and trying to connect theorist with different theories”. A third graduate also commented: “I enjoyed learning about the various theories, however, preparing and constructing my synthesis paper was a struggle for me. I mostly had trouble finding peer reviewed articles”. Table 4 The top five hardest courses in the online graduate program (number of responses=70) Top-rated courses Frequencies Percentage Theoretical Foundations of Educational Technology 22 31.4% Evaluation for Educational Technologists 18 25.7% Portfolio 8 11.4% Managing Technology Integration in Schools 6 8.6% Instructional Design 5 7.1% Australasian Journal of Educational Technology, 2019, 35(5). 67 Strongest sense of community Finally, we were interested in which courses and instructors were identified as the most memorable in terms of sense of community at the end of their program. For instance, did a specific course content or course design help establish a strong sense of community, or was it more the instructor? Or perhaps it might be the communication technologies used in a certain course? Further, we were curious if there was a relationship between one’s favourite, easiest, or hardest course and sense of community. Sense of community is sometimes idealised in the literature as something all students want and need in their online courses. We posit that some students, and sometimes instructors, do not need a strong sense of community in the courses they take or teach but might also prefer less social and community-focused activities and/or find it difficult to engage in community building activities (cf. Cain, 2013; Godsey, 2015). For instance, Su, Bonk, Magjuka, Liu, and Lee (2005) reported that individual personality traits or learning styles affect students’ perceptions about the need for interaction in online courses. In another example, Liu et al. (2007) found that instructors’ beliefs related to online learning (flexibility, self-paced learning, students’ busy lives) affect their consideration about the relevance of sense of community. There was actually less agreement on identifying the course with the strongest sense of community than all of the previous questions. Close to 20% of the respondents (see Table 5) identified Introduction to Educational Technology as the course with the strongest sense of community. When asked to explain why they selected this course, 79% of the respondents described how the course consisted of multiple collaborative small group projects. For example, one graduate explained: We were required to work in a small team throughout the course and I was fortunate to have an amazing team. Also, the professor used a shared Google doc and would answer our individual questions instead of insisting on a community forum. Another graduate stated: “We were required to participate in a lot of forum discussions and it really made us become ‘real’ to each other”. A third respondent emphasised the importance of using Google+ for discussions: “We interacted on Google+ as well as discussions. We were assigned to other people to interact with in Google+ blogs and discussions”. The second most popular reason for the strong sense of community, mentioned by 14% of the respondents, was the instructor. Interestingly, this online course was taught by three different instructors, thus suggesting that the course design had more to do with the sense of community (or the fact that this course is often taken during the first semester of a student’s program) than the instructor. A similar trend occurred with the Evaluation for Educational Technologies which was identified by almost 16% of the participants. However, in this case, the course was taught by the same instructor. Table 5 The top five courses with the strongest sense of community (number of responses = 69) Top-rated courses Frequencies Percentage Introduction to Educational Technology 13 18.8% Evaluation for Educational Technologists 11 15.9% Educational Games & Simulations 6 8.7% YouTube for Educators 6 8.7% The Internet for Educators 5 7.3% The participants completed Rovai’s (2002b) CCS based on the course they identified as having the strongest sense of community. When looking at the two courses identified as having the strongest sense of community, the courses scored almost the exact same on the CCS (Table 6). Though, both courses were ranked higher on the learning subscale than the connectedness subscale. Australasian Journal of Educational Technology, 2019, 35(5). 68 Table 6 Descriptive Statistic of the CCS Survey Items EDTECH 501 (n = 13) EDTECH 505 (n = 11) Classroom community 67.1 66.5 Connectedness subscale 31.4 31.4 Learning subscale 35.7 35.1 Note. Total possible classroom community scores range from 0 to 80, with higher scores reflecting a stronger sense of community. Connectedness and learning subscale scores can each range from 0 to 40. Discussions and implications We set out to learn more about what our students thought about their coursework once they graduated from the program. We were specifically interested in learning about our graduates’ overall satisfaction with the courses they completed (e.g., which courses were their favourite and why, which ones were the easiest and why, and which ones were the hardest and why) as well as the sense of community they experienced in their courses. We were also interested in whether student satisfaction and sense of community were more influenced by course content and course design or more by the course instructor. We began this study in hopes of finding ways to improve how we both design and teach the courses in our program. We use a master course type of model in our program where a lead faculty member designs courses that are then taught by other full-time and part-time instructors in the department throughout the year. Dominant quality assurance models in higher education, for example, Quality Matters, and specifically the quality assurance practices developed based on these frameworks suggest that quality online learning depends largely on how an online course is designed and less on how the course is taught (Baldwin et al., 2018; Baldwin & Trespalacios, 2017). However, research on higher education in general (e.g., Chickering & Gamson, 1987) and research on areas of interaction, social presence, connection, and community in online learning (e.g., Croxton, 2014) suggest that instructors play pivotal roles in online learning. The majority of the courses in our program are project-based and practical in nature. Thus, students in our program often have opportunities to use technology to develop lessons and/or instructional projects to use in their jobs. Therefore, it was not surprising to find that our graduates’ favourite courses in the MET program were Creating Educational Websites, Educational Games and Simulations, and Mobile App Design for Teaching and Learning, which all focus on innovative technologies for educators. It also makes sense that the fourth course most frequently cited as graduates’ favourite was Technology-Supported Project-based Learning. While it does not focus on innovative technologies, it is a very practical and useful course for educators. Thus, this suggests that students who choose to complete a degree in educational technology, enjoy hands-on courses focused on the practical use of innovative technologies. This finding supports the relevance of well-designed activities under the constructive or cooperative model (i.e., course dimension: e-learning course quality) to promote learners’ satisfaction (Sun et al., 2008). This finding could be also related to how student-content and student-student interactions influence student satisfaction (Bolliger & Martindale, 2004; Kuo, Walker, Belland, & Schroder, 2013). For instance, Swan (2002) found that students who reported high levels of interaction with the content in their courses also reported significantly higher levels of satisfaction and perceived learning in their online courses. The courses that our graduates reported were the hardest courses (i.e., Theoretical Foundations of Educational Technology, Evaluation for Educational Technologists, Portfolio, Managing Technology Integration in Schools) are in many ways the exact opposite of their favourite courses. The first two courses, selected by almost 60% of the respondents, focus on theory and academic reading and writing. Students and practitioners alike often fail to see the relevance or appreciate the value of research and theory (Dunlap & Lowenthal, 2013). However, leaders and professional standards in the field both highlight the importance for educational technologists to have knowledge, skills, and abilities focused on research and theory (AECT, 2012; Merrill, 2014). Therefore, programs, like ours, need to do a better job of helping our students Australasian Journal of Educational Technology, 2019, 35(5). 69 and graduates see the relevance and value of research and learning/instructional theory. One approach might be to find better ways to integrate research and theory into other courses and avoid having students take a course called Theoretical Foundations. However, perhaps we also need to find ways to help practitioners in the field see the value of research and theory. Introduction to Educational Technology (18.8%), Evaluation for Educational Technologists (15.9%), and Educational Games & Simulations (8.7%) were the courses with the highest sense of community. Graduates emphasised the relevance of collaborative projects that enabled and encouraged continuous interaction with their classmates and the instructor. As described by Sun et al. (2008), course dimensions (e-learning course quality), and environmental dimensions (learner perceived interaction with others) are relevant constructs to promote learners’ satisfaction. This finding is also in line with previous research that highlight the importance of designing activities in online courses that promote quality interactions as well focus on community building (Kreijns, Kirschner, & Jochems, 2003; Luo, Zhang & Qi, 2017; Ouzts, 2006; Rovai, 2002a; Swan, 2002; Shackelford & Maxwell, 2012) as well as how online group work can influence students’ sense of social presence and community (cf. Dunlap & Lowenthal, 2014). Although previous research has discussed the relationship between instructors and students’ satisfaction (Bolliger & Martindale, 2004; Sun et al., 2008; Swan, 2001), we were surprised to find that graduates in this study, when asked about their previous coursework, focused mostly on the content of the course, rather than their instructor. This does not mean that the quality of the instruction is not important in online courses but rather, that with this data, what students remember more about their online courses, after the fact, was the content of the course, not the instructor and how the instructor taught the course (i.e., the instructor dimension). Additionally, the data that we collected from our graduates suggests a possible third area of focus when it comes to creating and facilitating engaging and memorable learning experiences. This area is how the content or subject matter of the course, individual students’ interest in the course, and students’ choice to take the course play in creating high-quality learning experiences in online courses. However, even with considering this third perspective, the results of this study seem to suggest that memorable and satisfying online courses with a sense of classroom community are possibly a complex result of the content of the course, the students’ interest in the course, the instructional activities and course design, the collaboration/interaction, and the instructor of the course. Conclusions The number of online master’s degrees has grown dramatically in the past few years. With this growth has come increased competition between programs. Thus, online educators are interested in investigating the strengths and weaknesses of their programs in efforts to improve the student learning experience which in turn might help them be more competitive in the higher education marketplace. One way to investigate the strengths and weakness of a program is to reach out to those who have recently graduated from the program to understand what they liked and did not like as well as what was memorable and applicable in their day- to-day job. Students’ satisfaction is a key element to retain students in online programs. Educators need to design course activities that allow students to interact and collaborate in relevant activities to make their learning meaningful. Although graduates’ favourite courses are aligned with the needs of prospective employers looking for educational technologists with a strong set of multimedia production skills (Ritzhaupt, Martin & Daniels, 2010), it is important to emphasise that design, management, and communication skills are also relevant for future employment (Reiser, 2018). Based on Sun’s et al. (2008) learner dimension, we could possibly assume that graduates from our program in educational technology might have a positive attitude toward computers and internet self-efficacy that positively influence their satisfaction with their online learning environments. Thus, while the results of our study should not be generalised to represent all educational technology graduates, the results might have practical significance not only to other educational technology programs but other similar online professional master’s degrees. When reflecting on what they liked about our program, graduates focused mostly on the content of their coursework and specifically on courses that focused on the practical use of innovative and emerging technologies as opposed to courses focused on research and theory. They also focused less than expected on the role of the instructors in their courses, even when reflecting on their sense of community. To some degree, this could be the result of how the survey questions were worded and what Australasian Journal of Educational Technology, 2019, 35(5). 70 the graduates thought we wanted to know. However, it could also point to a need for instructors in our program to make a better effort to interact one-on-one with students and build relationships with students in the program. Additional research is needed to investigate the best ways to accomplish this. References Allen, J. D. (2005). Grades as valid measures of academic achievement of classroom learning. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 78(5), 218-223. https://doi.org/10.3200/TCHS.78.5.218-223 Angelino, L. M., Williams, F. K., & Natvig, D. (2007). Strategies to engage online students and reduce attrition rates. Journal of Educators Online, 4(2). Retrieved from http://www.thejeo.com/Volume4Number2/Angelino%20Final.pdf Association for Educational Communications and Technology (2012). AECT standards: 2012 version. Retrieved from https://c.ymcdn.com/sites/aect.site- ym.com/resource/resmgr/AECT_Documents/AECTstandards2012.pdf Baldwin, S., Ching, Y. H., & Hsu, Y. C. (2018). Online course design in higher education: A review of national and statewide evaluation instruments. TechTrends, 62(1), 46-57. Baldwin, S., & Trespalacios, J. H. (2017). Evaluation instruments and good practices in online education. Online Learning, 21(2). https://doi.org/10.24059/olj.v21i2.913 Barab, S. A., & Duffy, T. M. (2000). From practice fields to communities of practice. In D. H. Jonassen, & S. M. Land (Eds.), Theoretical foundations of learning environments (pp. 25-55). Mahwah, NJ: Lawrence Erlbaum. Baturay, M. H. (2011). Relationships among sense of classroom community, perceived cognitive learning and satisfaction of students at an e-learning course. Interactive Learning Environments, 19(5), 563- 575. https://doi.org/10.1080/10494821003644029 Bawa, P. (2016). Retention in online courses: Exploring issues and solutions - A literature review. Sage Open, 6(1), 1-11. https://doi.org/10.1177/2158244015621777 Bolliger, D. U., & Martindale, T. (2004). Key factors for determining student satisfaction in online courses. International Journal on E-Learning, 3(1), 61-67. Boston, W. E., Díaz, S. R., Gibson, A. M., Ice, P., Richardson, J., & Swan, K. (2009). An exploration of the relationship between indicators of the community of inquiry framework and retention in online programs. Journal of Asynchronous Learning Networks, 13(3), 67-83 Boston, W. E., Ice, P., & Gibson, A. M. (2011). Comprehensive assessment of student retention in online learning environments. Online Journal of Distance Learning Administration, 14(1). Retrieved from http://www.westga.edu/~distance/ojdla/spring141/boston_ice_gibson141.html Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience and school. Washington, DC: National Academy Press. Brown, A. L., & Campione, J. C. (1994). Guided discovery in a community of learners. In K. McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 229-270). Cambridge, MA: MIT Press. Brown, R. (2001). The process of community building in distance learning classes. Journal of Asynchronous Learning Networks, 5(2), 18–35. Cain, S. (2013). Quiet: The power of introverts in a world that can't stop talking. New York, NY: Broadway Books. Chang, Z. (2012). Student satisfaction, performance, and knowledge construction in online collaborative learning. Journal of Educational Technology and Society, 15(1), 127-136. Chickering, A. W., & Gamson, Z. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 40(7), 3-7. https://doi.org/10.1002/tl.37219914708 Creswell, J. W. (2015). Educational research. Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Upper Saddle River, NJ: Pearson. Croxton, R. A. (2014). The role of interactivity in student satisfaction and persistence in online learning. Journal of Online Learning and Teaching, 10(2), 314-324. Daugherty, M., & Funke, B. L. (1998). University faculty and student perceptions of web-based instruction. International Journal of E-Learning and Distance Education, 13(1), 21-39. DuFour, R., & Eaker, R. (1998). Professional learning communities at work: Best practices for enhancing student achievement. Bloomington, IN: National Educational Service. http://dx.doi.org/10.3200/TCHS.78.5.218-223 http://www.thejeo.com/Volume4Number2/Angelino%20Final.pdf https://c.ymcdn.com/sites/aect.site-ym.com/resource/resmgr/AECT_Documents/AECTstandards2012.pdf https://c.ymcdn.com/sites/aect.site-ym.com/resource/resmgr/AECT_Documents/AECTstandards2012.pdf https://doi.org/10.24059/olj.v21i2.913 https://doi.org/10.1080/10494821003644029 https://doi.org/10.1177/2158244015621777 http://www.westga.edu/%7Edistance/ojdla/spring141/boston_ice_gibson141.html http://dx.doi.org/10.1002/tl.37219914708 Australasian Journal of Educational Technology, 2019, 35(5). 71 Dunlap, J. C., & Lowenthal, P. R. (2013). What was your best learning experience? Our story about using stories to solve instructional problems. International Journal of Teaching and Learning in Higher Education, 25(2), 269-274. Dunlap, J. C., & Lowenthal, P. R. (2014). The power of presence: Our quest for the right mix of social presence in online courses. In A. A. Piña, & A. P. Mizell (Eds.), Real life distance education: Case studies in practice (pp. 41-66). Greenwich, CT: Information Age Publishing. Eom, S. B., Wen, H. J., & Ashill, N. (2006). The determinants of students’ perceived learning outcomes and satisfaction in university online education: An empirical investigation. Decision Sciences Journal of Innovative Education, 4(2), 215-235. Gannon‐Leary, P., & Fontainha, E. (2010). Communities of learning: Extending the boundaries of the learning experience through cross‐institutional collaboration. In J. M. Spector, D. Ifenthaler, P. Isaías, Kinshuk, & D. G. Sampson (Eds.), Learning and instruction in the digital age (pp. 237‐57). New York, NY: Springer. Garrett Dikkers, A., Whiteside, A. L., & Lewis, S. (2012). Get present: Build community and connectedness online. Learning & Leading with Technology, 40(2), 22-25. Godsey, M. (2015, September 9). When schools overlook introverts. The Atlantic. Retrieved from https://www.theatlantic.com/education/archive/2015/09/introverts-at-school-overlook/407467/ Haythornthwaite, C., Kazmer, M. M., Robins, J., & Shoemaker, S. (2000). Community development among distance learners: Temporal and technological dimensions. Journal of Computer-Mediated Communication, 6(1). https://doi.org/10.1111/j.1083-6101.2000.tb00114.x Hill, J. R. (2012). Learning communities: Theoretical foundations for making connections. In D. Jonassen, & S. Land (Eds.), Theoretical foundations of learning environments (pp. 269-285). New York, NY: Routledge. Hill, P. (1985, October). The rationale for learning communities and learning community models. Paper presented at the inaugural conference on learning communities of the Washington Center for Improving the Quality of Undergraduate Education, Olympia, WA. Retrieved from http://files.eric.ed.gov/fulltext/ED309818.pdf Johnson, V. E. (2006). Grade inflation: A crisis in college education. New York, NY: Springer. Jonassen, D. H. (1995). Supporting communities of learners with technology: A vision for integrating technology with learning in schools, Educational Technology, 35(4), 60–63. Kear, K. (2011). Online and social networking communities: A best practice guide for educators. New York, NY: Routledge. Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior 19, 335–353. https://doi.org/10.1016/S0747-5632(02)00057-2 Kuo, Y-C., Walker, A. E., Belland, B. R., & Schroder, K. E. (2013). A predictive study of student satisfaction in online education programs. The International Review of Research in Open and Distance Learning, 14(1), 16-39. https://doi.org/10.19173/irrodl.v14i1.1338 Kuo, Y-C., Walker, A. E., Schroder, K. E., & Belland, B. R. (2013). Interaction, internet self-efficacy, and self-regulated learning as predictors of student satisfaction in online education courses. Internet and Higher Education, 20, 35-50. https://doi.org/10.1016/j.iheduc.2013.10.001 Lally, V. & Barrett, E. (1999). Building a learning community on-line: Towards socio-academic interaction. Research Papers in Education, 14(2), 147. Retrieved from https://www.learntechlib.org/p/89251/ Lawless, C., & Richardson, J. T. (2004). Monitoring the experiences of graduates in distance education. Studies in Higher Education, 29(3), 353-374. https://doi.org/10.1080/03075070410001682628 Liu, X., Magjuka, R. J., Bonk, C. J., & Lee, S. (2007). Does sense of community matter? Quarterly Review of Distance Education, 8, 9–24. Lowenthal, P. R. (2010). The evolution and influence of social presence theory on online learning. In T. T. Kidd (Ed.), Online education and adult learning: New frontiers for teaching practices (pp. 124- 139). Hershey, PA: IGI Global. Lowenthal, P. R., Bauer, C., & Chen, K-Z. (2015). Student perceptions of online learning: An analysis of online course evaluations. American Journal of Distance Education, 29(2), 85-97. https://doi.org/10.1080/08923647.2015.1023621 Ludwig-Hardman, S., & Dunlap, J. C. (2003). Learning support services for online students: Scaffolding for success. International Review of Research in Open and Distance Learning, 4(1), 1-15. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/131/211 https://www.theatlantic.com/education/archive/2015/09/introverts-at-school-overlook/407467/ http://dx.doi.org/10.1111/j.1083-6101.2000.tb00114.x http://files.eric.ed.gov/fulltext/ED309818.pdf http://dx.doi.org/10.1016/S0747-5632(02)00057-2 http://dx.doi.org/10.19173/irrodl.v14i1.1338 http://dx.doi.org/10.1016/j.iheduc.2013.10.001 https://www.learntechlib.org/p/89251/ http://dx.doi.org/10.1080/03075070410001682628 http://dx.doi.org/10.1080/08923647.2015.1023621 http://www.irrodl.org/index.php/irrodl/article/view/131/211 Australasian Journal of Educational Technology, 2019, 35(5). 72 Luo, N., Zhang, M., & Qi, D. (2017). Effects of different interactions on students' sense of community in e-learning environment. Computers & Education, 115, 153-60. https://doi.org/10.1016/j.compedu.2017.08.006 Luppicini, R. (Ed.). (2007). Online learning communities. Charlotte, NC: Information Age Publishing. McDonald, J., & Cater-Steel, A. (Eds.). (2017). Communities of practice: Facilitating social learning in higher education. Singapore: Springer. McGorry, S. Y. (2003). Measuring quality in online programs. The Internet and Higher Education, 6(2), 159–177. https://doi.org/10.1016/S1096-7516(03)00022-8 Merrill, M. D. (2014). My hopes for the future of instructional technology. Educational Technology, 54(4), 22-26. Moore, J. C. (Ed.). (2002). Elements of quality: The Sloan-C framework. Needham, MA: Sloan Center for Online Education. Ni, S-F., & Aust, R. (2008). Examining teacher verbal immediacy and sense of classroom community in online classes. International Journal on E-Learning, 7(3), 477-498. Ouzts, K. (2006). Sense of community in online courses. Quarterly Review of Distance Education, 7(3), 285–296. Palloff, R., & Pratt, K. (2007). Building online learning communities: Effective strategies for the virtual classroom. San Francisco, CA: Jossey-Bass. Park, J. H., & Choi, H. J. (2009). Factors influencing adult learners' decision to drop out or persist in online learning. Journal of Educational Technology & Society, 12(4), 207-217. Retrieved from https://search.proquest.com/docview/1287038599?pq-origsite=gscholar Patterson, B., & McFadden, C. (2009). Attrition in online and campus degree programs. Online Journal of Distance Learning Administration, 12(2). Retrieved from http://www.westga.edu/~distance/ojdla/summer122/patterson112.html Picciano, A. G. (2002). Beyond student perceptions: Issues of interaction, presence, and performance in an online course. Journal of Asynchronous Learning Networks, 6(1), 21-40. Reiser, R. A. (2018). Getting an instructional design position: Lessons from a personal history. In R. A. Reiser, & J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (4th ed., pp. 207-212). New York, NY: Pearson Education. Richardson, J. C., Maeda, Y., Lv, J., & Caskurlu, S. (2017). Social presence in relation to students' satisfaction and learning in the online environment: A meta-analysis. Computers in Human Behavior, 71, 402-417. Ritzhaupt, A., Martin, F. & Daniels, K. (2010). Multimedia competencies for an educational technologist: A survey of professionals and job announcement analysis. Journal of Educational Multimedia and Hypermedia, 19(4), 421-449. Retrieved from https://www.learntechlib.org/p/34114/. Rogers, G. (2003). Do grades make the grade for program assessment? ABET Assessment Planning. Retrieved from http://www.abet.org/wp-content/uploads/2015/04/do-grades-make-the-grade.pdf Rogoff, B. (1994). Developing understanding of the idea of communities of learners. Mind, Culture, and Activity, 1(4), 209-229. Rovai, A. P. (2001). Building classroom community at a distance: A case study. Educational Technology Research and Development Journal, 49(4), 35–50. https://doi.org/10.1007/BF02504946 Rovai, A. P. (2002a). Building sense of community at a distance. International Review of Research in Open and Distance Learning, 3(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/79 Rovai, A. P. (2002b). Development of an instrument to measure classroom community. Internet and Higher Education, 5(3), 197–211. https://doi.org/10.1016/S1096-7516(02)00102-1 Rovai, A. P. (2003). A practical framework for evaluating online distance education programs. Internet and Higher Education, 6(2), 109-124. https://doi.org/10.1016/S1096-7516(03)00019-8 Rovai, A. P., & Baker, J. D. (2005). Gender differences in online learning. Quarterly Review of Distance Education, 6(1), 31–44. Rovai, A. P., Ponton, M. K., & Baker, J. D. (2008). Distance learning in higher education: A programmatic approach to planning, design, instruction, evaluation, and accreditation. New York, NY: Columbia University Press. Seaman, J. E., Allen, I. E., & Seaman, J. (2018). Grade increase. Tracking distance education in the United States. Retrieved from https://onlinelearningsurvey.com/reports/gradeincrease.pdf Shackelford, J. L., & Maxwell, M. (2012). Sense of community in graduate online education: Contribution of learner to learner interaction. International Review of Research in Open and Distance Learning, 13(4), 228-248. https://doi.org/10.19173/irrodl.v13i4.1339 http://dx.doi.org/10.1016/j.compedu.2017.08.006 http://dx.doi.org/10.1016/S1096-7516(03)00022-8 https://search.proquest.com/docview/1287038599?pq-origsite=gscholar http://www.westga.edu/%7Edistance/ojdla/summer122/patterson112.html https://www.learntechlib.org/p/34114/ http://www.abet.org/wp-content/uploads/2015/04/do-grades-make-the-grade.pdf http://dx.doi.org/10.1007/BF02504946 http://www.irrodl.org/index.php/irrodl/article/view/79 http://dx.doi.org/10.1016/S1096-7516(02)00102-1 http://dx.doi.org/10.1016/S1096-7516(03)00019-8 https://onlinelearningsurvey.com/reports/gradeincrease.pdf http://dx.doi.org/10.19173/irrodl.v13i4.1339 Australasian Journal of Educational Technology, 2019, 35(5). 73 Shea, P. (2006). A study of students' sense of learning community in online environments. Journal of Asynchronous Learning Networks, 10(10). http://onlinelearningconsortium.org/sites/default/files/v10n1_4shea.pdf Shea, P., & Bidjerano, T. (2014). Does online learning impede degree completion? A national study of community college students. Computers & Education, 75, 103-111. https://doi.org/10.1016/j.compedu.2014.02.009 Shea, P., Li, C. S., Swan, K., & Pickett, A. (2005). Developing learning community in online asynchronous college courses: The role of teaching presence. Journal of Asynchronous Learning Networks, 9(4), 59-82. Sloan Consortium (2018). The 5 Pillars. Retrieved from https://onlinelearningconsortium.org/5-pillars/ Sloman, M., & Reynolds, J. (2003). Developing e-learning community. Human Resource Development International, 6(2), 259-272. https://doi.org/10.1080/13678860210155412 Stoll, L., Bolam, R., McMahon, A., Wallace, M., & Thomas, S. (2006). Professional learning communities: A review of the literature. Journal of Educational Change, 7(4), 221-258. https://doi.org/10.1007/s10833-006-0001-8 Su, B., Bonk, C. J., Magjuka, R. J., Liu, X., & Lee, S. (2005). The importance of interaction in web-based education: A program-level case study of online MBA courses. Journal of Interactive Online Learning, 4(1), 1-18. Sun, P.-C., Tsai, R. J., Finger, G., Chen, Y.-Y., & Yeh, D. (2008). What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4), 1183–1202. https://doi.org/10.1016/j.compedu.2006.11.007 Swan, K. (2001). Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses. Distance Education, 22(2), 306-331. https://doi.org/10.1080/0158791010220208 Swan, K. (2002). Building learning communities in online courses: The importance of interaction. Education, Communication & Information, 2(1), 23-49. https://doi.org/10.1080/1463631022000005016 Swan, K., & Shea, P. (2005). The development of virtual learning communities. In. S. R. Hiltz & R. Goldman (Eds.), Asynchronous learning networks: The research frontier (pp. 239-260). New York, NY: Hampton Press. Swan, K., Shea, P., Fredericksen, E., Pickett, A., Pelz, W., & Maher, G. (2000). Building knowledge building communities: Consistency, contact and communication in the virtual classroom. Journal of Educational Computing Research, 23(4), 359-383. https://doi.org/10.2190/W4G6-HY52-57P1-PPNE Tight, M. (2012). Researching higher education (2nd ed.). New York, NY: McGraw-Hill. Vlachopoulos, D. (2016). Assuring quality in e-learning course design: The roadmap. The International Review of Research in Open and Distributed Learning, 17(6). https://doi.org/10.19173/irrodl.v17i6.2784 Vogt, F. D., & Slish, D. (2011). A model for programmatic evaluation by student assessment. Journal of Science Education and Technology, 20(6), 796-802. https://doi.org/10.1007/s10956-010-9274-z Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York, NY: Cambridge University Press. Wenger, E. (2000). Communities of practice and social learning systems. Organization, 7(2), 225-246. https://doi.org/10.1177/135050840072002 Whiteside, A. L. (2015). Introducing the Social Presence Model to explore online and blended learning experiences. Online Learning Journal, 19(2). Retrieved from https://olj.onlinelearningconsortium.org/index.php/olj/article/view/453/137 Whiteside, A. L., Garrett Dikkers, A., & Lewis, S. (2014, May 19). The power of social presence for learning. EDUCAUSE Review Online. Retrieved from https://er.educause.edu/articles/2014/5/the- power-of-social-presence-for-learning Wilson, B. G., Ludwig-Hardman, S., Thornam, C. L., & Dunlap, J. (2004). Bounded community: Designing and facilitating learning communities in formal courses. International Review of Research in Open and Distance Learning, 5(3). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/204/286 Winters, T., & Payne, T. (2005). What do students know? An outcomes-based assessment system. In R. Anderson, S. A. Fincher, & M. Guzdial (Eds.), Proceedings of the first International Computing Education Research Workshop (pp. 165-172). New York, NY: Association for Computing Machinery. http://onlinelearningconsortium.org/sites/default/files/v10n1_4shea.pdf https://doi.org/10.1016/j.compedu.2014.02.009 https://onlinelearningconsortium.org/5-pillars/ http://dx.doi.org/10.1080/13678860210155412 http://dx.doi.org/10.1007/s10833-006-0001-8 https://doi.org/10.1016/j.compedu.2006.11.007 http://dx.doi.org/10.1080/0158791010220208 http://dx.doi.org/10.1080/1463631022000005016 http://dx.doi.org/10.2190/W4G6-HY52-57P1-PPNE https://doi.org/10.19173/irrodl.v17i6.2784 http://dx.doi.org/10.1007/s10956-010-9274-z http://dx.doi.org/10.1177/135050840072002 https://olj.onlinelearningconsortium.org/index.php/olj/article/view/453/137 https://er.educause.edu/articles/2014/5/the-power-of-social-presence-for-learning https://er.educause.edu/articles/2014/5/the-power-of-social-presence-for-learning http://www.irrodl.org/index.php/irrodl/article/view/204/286 Australasian Journal of Educational Technology, 2019, 35(5). 74 Yang, D., Baldwin, S., & Snelson, C. (2017). Persistence factors revealed: Students’ reflections on completing a fully online program. Distance Education, 38(1), 23-36. https://doi.org/10.1080/01587919.2017.1299561 Young, S. (2006). Student views of effective online teaching in higher education. The American Journal of Distance Education, 20(2), 65-77. https://doi.org/10.1207/s15389286ajde2002_2 Corresponding author: Jesus Trespalacios, jesustrespalacios@boisestate.edu Australasian Journal of Educational Technology © 2019. Please cite as: Trespalacios, J., & Lowenthal, P. R. (2019). What do they really like? An investigation of students' perceptions of their coursework in a fully online educational technology program. Australasian Journal of Educational Technology, 35(5), 60-78. https://doi.org/10.14742/ajet.4363 https://doi.org/10.1080/01587919.2017.1299561 http://dx.doi.org/10.1207/s15389286ajde2002_2 mailto:jesustrespalacios@boisestate.edu https://doi.org/10.14742/ajet.4363 Australasian Journal of Educational Technology, 2019, 35(5). 75 Appendix A Educational Technology Program Profile Mission Statement Educational Technology is an innovative way to design, deliver, facilitate, and manage instruction for learners of all ages, whether it is face-to-face in a classroom, online, or a combination of methods. EdTech is much more than multimedia add-ons. It’s a way of engaging learners so they can learn academic content using interesting 21st century methods and tools. ● Empower learners to be evolving digital citizens who advocate cultural understanding and global responsibility. ● Promote, manage, and evaluate digital-age teaching and learning resources in K-higher education environments. ● Promote and pattern participatory culture, professional practice, and lifelong learning. • Design and develop imaginative learning environments. ● Lead research and innovations in online teaching and learning. ● Forge connections between research, policy, and practice in educational technology. ● Inspire creativity and expertise in digital media literacies. M.E.T Program organization The M.E.T. is a practitioner-focused program for educators planning on the master’s as their terminal degree. Students need 15 credits on core classes, 15 credits on elective courses, and 3 additional credits on a culminating activity that includes the elaboration of a professional portfolio. Course Title Course Description Credits Core Courses Introduction to Educational Technology Overview of the field of educational technology emphasizing current issues, leadership in technology use planning, and evaluation/synthesis of research. 15 The Internet for Educators Design and develop instructional web pages using HTML, CSS, and Adobe Dreamweaver. Apply instructional strategies when creating educational websites while taking into consideration issues of copyright and accessibility. Instructional Design for Educators Focuses on systematic design of instruction and alternative models. Project required. Theoretical Foundations for Educational Technology Overview of classic and contemporary theories of learning and their applications in educational technology and emerging orientations; implications for practice. Evaluation for Educational Technologists Procedures for evaluating educational programs, training systems, and emergent-technology applications. Examples of Elective Courses Online Course Design Emphasizes web-based instructional design for the development of online courses. Consideration is given to Australasian Journal of Educational Technology, 2019, 35(5). 76 various models of online delivery, content organization and presentation, and graphic design. Course participants create a fully developed online course. 15 Educational Games and Simulations Explores the theory and implementation of educational games, simulations, and virtual environments for improved instructional engagement. Includes evaluation methods and socio-cultural implications. Graphic Design for Learning Apply learning theory, principles of visual literacy, and graphic design techniques for instructional media development. Select and combine visual and text representations to effectively communicate instructional information. Mobile App Design for Teaching and Learning Students leverage the potential of mobile technologies by exploring, analyzing, and designing mobile apps for use in various settings such as teaching, learning, and work. Technology-supported Project-based Learning Examines the Project-Based Learning Model, including development of PBL-based instructional units that engage learners in projects requiring investigation, analysis, synthesis, and presentation in real-world scenarios. YouTube for Educators Produce educational video for YouTube using digital video cameras and editing software. Design and develop appropriate instructional activities that integrate online video. Examine the benefits and controversial aspects of video sharing in the classroom. Culminating Activity Portfolio Based on the AECT standards, portfolio is a broad-based selection of significant student work that is used to appraise student performance and professional development. A portfolio reflects the depth and breadth of a student’s educational growth since entering the graduate program. Portfolios may include, but are not limited to, classroom examinations, journals, writing samples, publishable scholarship, professional projects, annotated bibliographies, and artistic endeavors. 3 TOTAL 33 Australasian Journal of Educational Technology, 2019, 35(5). 77 Appendix B Survey We are investigating classroom community and therefore are interested in your experience in the Master of Educational Technology (MET) program. Some courses focus more on developing a classroom community than others. The questions below are meant to help us better understand what courses and what instructional strategies are related to classroom community development. Read each statement or question carefully and select the statement that comes closest to how you feel about the course. There are no correct or incorrect responses. 1. My favorite course in the program was: 2. This course was my favorite because: 3. The easiest course in the program was: 4. I found this course easy because: 5. The hardest course in the program was: 6. I found this course hard because: 7. It is important to feel connected to others in online courses: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 8. It is important to get a sense that other students are “real” and “there” (i.e., their social presence) in online courses: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 9. It is important to get a sense that instructors are “real” and “there” (i.e., their social presence) in online courses: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 10. I sometimes felt isolated or alone in the online courses I completed in the program: [(strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 11. When I think about the courses I took in the MET program and of classroom community in online courses, the following course had the strongest sense of classroom community: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 12. This course was taught by the following instructor (note: identifying the instructor will help us identify the course and what strategies helped develop community). If you don't see the instructor listed, please type in the name of the instructor in the next question: 13. This course had a strong sense of classroom community because: Classroom Community Scale The following 20 questions are focused on the course that you took that you felt had the strongest sense of classroom community. If you feel that there has been too much time that has passed since you took the course, skip to the last question. 14. When reflecting on this course, I felt that students in this course cared about each other: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 15. When reflecting on this course, I felt encouraged to ask questions: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 16. When reflecting on this course, I felt connected to others in the course: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 17. When reflecting on this course, I felt that it was hard to get help when I had a question: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 18. When reflecting on this course, I did not feel a spirit of community: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 19. When reflecting on this course, I felt that I received timely feedback: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 20. When reflecting on this course, I felt that the course was like a family: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 21. When reflecting on this course, I felt uneasy exposing gaps in my understanding [Strongly Disagree - -- Strongly Agree] 22. When reflecting on this course, I felt isolated in the course: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 23. When reflecting on this course, I felt reluctant to speak openly: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) Australasian Journal of Educational Technology, 2019, 35(5). 78 24. When reflecting on this course, I trusted others in the course: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 25. When reflecting on this course, I felt that the course resulted in only modest learning: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 26. When reflecting on this course, I felt that I could rely on others in this course: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 27. When reflecting on this course, I felt that other students did not help me learn: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 28. When reflecting on this course, I felt that members of this course depended on me: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 29. When reflecting on this course, I felt that I was given ample opportunities to learn: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 30. When reflecting on this course, I felt uncertain about others in the course: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 31. When reflecting on this course, I felt that my educational needs were not being met: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 32. When reflecting on this course, I felt confident that others would support me: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) 33. When reflecting on this course, I felt that the course did not promote a desire to learn: (strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) Introduction Background Students’ satisfaction Sense of community Methodology Participants and context of the study Data collection and analysis Results Favourite online course Easiest online course Hardest online course Strongest sense of community Discussions and implications Conclusions