3893 Journal of Teaching and Learning with Technology, Vol. 13, No. 2, December 2013, pp. 1 – 4. Invited Essay MOOCs and me Donald A. Coffin1 Abstract: This invited essay explores one emeritus faculty members experience as a student in a MOOC. Keywords: MOOC, learning, elearning, massive open online course I recently enrolled in, and completed, a MOOC, partly because I have been following the discussion about them (I have, at the moment, compiled a library of 27 articles, blog posts, and other materials that have appeared since the middle of April), but perhaps primarily because I had volunteered to make a presentation about MOOCs to an organization of which I am a member.2 Our meeting was in mid-May, and I thought I would be able to talk more intelligently about MOOCs in general if I had some experience with at least one of them. My plan was to enroll in a course outside my discipline (economics), and in which I was interested, so that I would be able to experience the course as a student, not as someone with a considerable store of knowledge. Unfortunately, given my time frame, I was unable to find such a course that began in early-to-mid April. I wound up in a course that explored the economic development of the world from roughly the mid-18th century to the present. The course, which began near the end of April, ran through about the end of June. According to an end-of course communication from the instructor, 28,922 people enrolled. During the eight weeks of the course, the instructor presented eleven video lectures, which were subdivided into a total of 63 individual videos, which totaled about 26 hours and 40 minutes of material.3 In general, each section of the lectures (all 63 of them) had one or more ungraded learning activity for which sample answers were provided.4 The lectures, which were well- organized, well-sourced, and generally well-presented, were (essentially) a talking head accompanied by Power Point slides. Copies of the slides were available as .pdf documents. Generally, an un-graded learning activity accompanied each video, with suggested answers. According to the end-of-course communication to the class from the instructor, “there were 164,946 viewings” of the lectures, or an average of over 2,600 for each lecture part. In addition we were provided with an extensive (25 page) list of suggested readings, of which over 100 items were available on-line. I obviously do not know how much anyone else read, but (given that I am actually quite interested in this material) I downloaded what was available and have so far read about half of it. The course also had, for each week’s lectures, on-line discussion forums. The forums were created (and their topics determined) by the enrollees; in general, the discussions were wholly among the students, although the instructor and a couple of assistants would add 1 Emeritus Associate Professor of Economics, School of Business and Economics, Indiana University Northwest, dcoffin@iun.edu 2 Indiana University’s Faculty Colloquium on Excellence in Teaching. 3 In a typical 15 week semester for a class meeting 150 minutes per week, and (in the case of a typical class that I would teach) losing 150 minutes for in-class tests, there would be about 35 hours of class time, to be used for all in-class activity. So this looks to be roughly equivalent in terms of time to a regular semester course. 4 All this is from the course website. Coffin, D.A. Journal of Teaching and Learning with Technology, Vol. 13, No. 2, December 2013. jotlt.indiana.edu 2 comments that dealt with issues that the enrollees really could not provide answers for. The discussions were generally civil and well-conducted, although my estimate is that well over half the posts were personal opinions of the posters, often with little support beyond the personal experience of the poster. Most posters did so using their own names, which I think contributed to the civility of the conversations. We were informed at the end of the course that there were 423 discussion threads, with a total of 2,370 posts and 1,413 comments. So the discussions were fairly active.5 But, as usual, a small number of enrollees accounted for most of the activity. Based on the list of most active discussion forum participants, I calculate that only about 90 people were really active in the forums; they created about 44% of the threads and made about 44% of the posts; the 10 most active posters accounted for about 19% of the posts. (As I note below, only about 500 people completed all of the course activities.) (I don’t completely trust this, because this includes people who created 0 or 1 threads and made even a single post. I don’t see how it’s possible to be less active than that.) There were three peer-assessed activities; if one submitted a response to the activity, one was expected to grade a minimum of three posts by fellow-students. Many participants graded more than three. Two of these had word limits (activity 2 had a limit of 1,000 words and activity 3 had a limit of 1,500), which were widely ignored.6 We were provided with rubrics for assessing the work, but I’m not sure that they were detailed enough. Also, we were not provided with examples of what an assessment should look like.7 Grading was on a 1 (inadequate) to 4 (outstanding) scale; to receive a Certificate of Completion, one had to complete all three activities with a minimum score of 2. While there was clearly a large variance in how people approached grading, most of the feedback that I received was reasonable, and included comments that I agreed with about the strengths and weaknesses of what I submitted. My judgment is that the course content is similar to what would be provided in a campus—based course, either in-class or on-line. But the assessment of learning was, I think, wholly inadequate for course credit to be awarded for the class in any college or university. The inadequacy of the learning assessment is partly the extremely limited amount of the course material for which assessments were done, but also because of the peer assessment system. As I noted above, my experience with it was fairly positive. But, in fact, the assessment was handled by people who were not fully capable of assessing the validity of the arguments made in the activities. So as hard as people worked, it is unreasonable to expect them to have been able to provide the kind of assessments and feedback that someone with extensive disciplinary knowledge could provide. And how did the students in the course do? Again, based on the end-of-course information provided by the instructor, out of the (roughly) 29,000 registrants, only 12,917 did anything in the course, only about 700 completed the first activity, and only about 500 completed the second and third activities. We were told that the average scores on activities 1 & 2 were 2.86 and 3.06, respectively; no data were provided for the third activity. At most, then, the completion rate was somewhere under 2% of registrants and under 4% of those who participated at all. Granted, in a free course with no real payoff at the end (unless you think a 5 The most active thread appears to have been one of the gold standard (which I initiated), with 112 posts. 6 My own judgment is that these limits were considerably too strict. 7 My approach was to list all of the items in each rubric and “score” each assignment of each component of the rubric. I then tried to provide fairly detailed comments based on each part of the rubric. I generally spent somewhere around 30 minutes of each submission that I graded. (I wrote to the instructor to tell him that I thought people needed more guidance on assessing student work, and included a description of what I did, once the course was over. At that time, I also told him why I had been in the class and what my background was.) Coffin, D.A. Journal of Teaching and Learning with Technology, Vol. 13, No. 2, December 2013. jotlt.indiana.edu 3 Certificate of Completion is a payoff), we can expect a lot of “drive-by” registrants. But it does suggest that encouraging and maintaining effort in such a course will be an even greater problem than it is in campus-developed and campus-run online courses. What can I take away from my experience in this one course for an assessment of MOOCs in general? First, at the current state of development, attempting to use an existing MOOC as a substitute for a course developed and managed on campus is clearly not a good idea.8 Based on my experience, I would suggest that, at a minimum, using a MOOC as the basis for a credit-bearing course will require a strong on-campus component—a supervising faculty member on-campus, locally-designed (and graded) learning assessments, locally-operated and staffed discussion sections or on-line discussion forums (depending on the institution, the discussion groups/forums could be staffed by graduate teaching assistants, advanced undergraduate students, or a mix of the two. Whatever the mix, providing adequate training to the discussion leaders seems to me to be an essential component of making this work.) Using a MOOC in this way, of course, will not lead to the sort of cost savings that is one of the strongest lures for university administrators. In addition, even in introductory-level courses, whatever on- line component the course has will need to be updated often; in the case of (for example) an introductory macroeconomics class, annual updates would almost certainly be necessary because the macro economy will get you otherwise. This also reduces any cost savings that may occur. But done this way, I can easily see MOOCs playing a major role in courses now being taught on- campus in large lecture sections. There is, it seems to me, little difference between a talking head with Power Points on video and a talking head with Power Points at the front of a large classroom. Whether this means outsourcing course content (and learning objectives), or keeping control and creating your own MOOC, is the issue, I think. Done correctly, MOOCs may well kill the large lecture class. It is also clear that, judiciously used, MOOCs might be able to add a dimension to existing courses that continue to meet in classrooms or operate as single-campus online courses. This, of course, will work only if the schedule of the MOOC and the class schedule can be synchronized. An alternative is to take the power of video and online instruction much more seriously than many people seem to do. Indeed, I have yet to see a MOOC that seriously goes beyond an online talking head (but I have not seen even a significant fraction of everything that is currently available.) Alex Tabarrok, one of the creators of a MOOC site emphasizing economics courses, has written that “To take full advantage of the online format, an online lecture has to be different from an in-class lecture. Different mediums demand different messaging.”9 Doing this—finding and incorporating interesting, informative images, sound, and video clips—would surely enhance the educational process. But it would also be likely to increase considerably the cost of producing the videos for an online course.10 8 Recent commentary on the experience at San Jose State University would suggest that at least one effort to use an existing MOOC as a substitute for on-campus courses has not worked out well http://chronicle.com/blognetwork/tenuredradical/2013/07/f-is-for-failure-or-dont-invest-your-pension-in-moocs-yet/ and http://www.slate.com/blogs/future_tense/2013/07/19/san_jose_state_suspends_udacity_online_classes_after_students_fail_final. html) . 9 “Why Online Education Works,” Cato Unbound (http://www.cato-unbound.org/2012/11/12/alex-tabarrok/why-online- education-works). In an ironic twist, the course that Tabarrok and one of his colleagues (Tyler Cowen) developed for their MOOC site (Marginal Revolution University, at http://mruniversity.com/) uses still images with voice-over narration. While the images are often appealing, they do not add information to the process and are static. 10 Such material is often available on publisher websites as supplements to textbooks. My conversations with publishers’ representatives suggests that these are expensive to produce (including rights fees) and to keep current). Coffin, D.A. Journal of Teaching and Learning with Technology, Vol. 13, No. 2, December 2013. jotlt.indiana.edu 4 Finally, I would encourage anyone who wonders about the implications of MOOCs for higher education to enroll in one, participate fully in it, and see how it works, either for something you want to learn about or for your own discipline. It’s clear that they are not going away soon, and knowing, from the inside, how they work—and don’t work—will be an essentially part of making sure that, however they wind up being incorporated into our lives, it will not be any worse for us than it has to be.