AN EVALUATION OF A DISTRIBUTED LEARNING COURSE: A STUDENTS’-EYE PERSPECTIVE Developments in Business Simulation and Experiential Learning, Volume 31, 2004 AN EVALUATION OF A DISTRIBUTED LEARNING COURSE: A STUDENTS’-EYE PERSPECTIVE J. Alexander Smith Wayne State University ae4651@wayne.edu ABSTRACT: But the term “Online Learning” does not cover all the bases, nor do the words “Distance Education” accurately describe this type of learning. The matter is further confused with the introduction of new terms like “E-learning”, “Web-Based Instruction” and “Virtual Classroom”. Each of these terms describe a single aspect of the new teaching techniques or use of technology, but are often flaunted as general terms that are supposed to encompass the entire spectrum of possibilities. To remedy this situation, the term “Distance Education” has been broadened to “Distributed Learning”. It includes both education where the teacher and the student are at different locations or time (Distance Education) as well as classes that may be traditional in nature, but are augmented by online tools and technology. Today there is increasing pressure to utilize the Internet and other new technologies to enhance and sometimes replace the classroom experience. With this recent push, professors are using new technologies simply because they are available without understanding how to use them effectively. For those that are new to these technologies, the learning curve can often be steep and dangerous. In order to remove some of these stumbling blocks, this paper presents an evaluation of a distributed learning experience. KEYWORDS: Distributed Learning, online learning, distance education To further clarify this idea, the Institute for Distributed Education of the University System of Maryland has defined three models of Distributed Learning (IDE 1997). These models are base upon the factors of Time and Place. Sharma Pillutla (2000) sums up those models as: INTRODUCTION The last decade has seen a dramatic change in technology. The push for new technology has seen a change in the way that we do business in the world. It has caused the rise and fall of corporate giants who were once acclaimed as being technologically advanced. Many of these new technologies have made their way into the classroom setting. With this surge of technological might, instructors have begun to use these new inventions in hopes of improving their teaching methods. Many are using them simply because they are available with no clear idea of what purpose they fill. (a) Distributed Classroom: Extending a traditional classroom using interactive technologies – time and place dependent. (b) Independent Learning: Contact between students and faculty is maintained by telephone, voice-mail, computer conferencing, electronic and regular mail – time and place independent (c) Open Learning + Class: Self-paced learning through occasional contact using interactive telecommunications technologies – place independent. Probably the most powerful of these new technologies has been the introduction of the Internet, often referred to as the World Wide Web (WWW). This technology is not exactly new, but it has only been during the last decade that it has been made accessible to the general public. In the few short years that it is been around, it has gained acceptance by the general populace faster than any previous technological advancement, including television and radio. The different uses for the Internet are limitless and are only beginning to be developed. Pillutla then adds a fourth model: (d) Extended Distributed Learning: extending a traditional classroom so that some portions are time and place dependent and others are time and place independent. It is this fourth model that applies most directly to our purposes today. We are currently in a stage of development where instructors of all levels are pursuing means to enhance their classroom instruction. When used properly, web technologies can be used to do more than simply deliver content. Instead of being the medium of delivery, the proper use of the web can increase learning. At least that is what we have been told. Jerry Gosen (2003) points out that, “the research available dealing with online education yields few conclusions, and many of the … studies are methodologically flawed.” If that is the case, then why are the universities pushing for new technologies almost faster than they are being produced? Perhaps it is the creation of the Internet, or the changes that it has caused that has brought about the increased demand for Distance Education. Once ignored by mainstream academia, it has gained widespread acceptance. Universities around the world are trying to find ways to implement Distance Education. New universities are being created to supply the growing demand for more accessible education. Many of these programs rely upon the Internet as their backbone. “Online learning” has become a popular buzzword. It has a wide definition that includes the use of the Internet as a tool for delivery and as a course enhancer. 40 mailto:ae4651@wayne.edu Developments in Business Simulation and Experiential Learning, Volume 31, 2004 EVOLUTION OF ONLINE INSTRUCTION Just as we have seen an evolution of business uses on the web, we have seen a similar process occurring with professor use of the web. Initially businesses used the web as a way to easily distribute sales literature. Called Brochureware, these websites allow consumers to view and compare product information with ease but provide limited means of interaction. Similarly professors used the web initially as a means to distribute notes and announcements. This idea becomes more obvious when we look at the elementary educators who are now posting assignments on class websites. These new websites allow parents to keep track of what their children are learning and allows children that stay home sick to keep up with class progress. This is a huge step in increasing the communication from instructor to student. Then the transactional websites began to take shape and the businesses could take orders and exchange funds with the consumer. It still provided limited direct contact with each other, though companies began relying on e-mail as their main tool of contact. Universities likewise started introducing transactional courses. In these situations the student could often find information, take a test, or upload course assignments with the instructor. Discussion boards became commonplace and new courses appeared trying to take advantage of the new transactional ability. Noriko Hara and Rob Kling (2000) published a qualitative case study of one such online course. Alvin Burns (1998) also presented comments on his initial attempt to teach a course solely through the web. Both were plagued with technical problems and student confusion. The next phase of evolution is the full-service model. In this phase, businesses utilize the Internet for the entire sales process of pre- and post-sales as well as the actual transaction. These sites are smart enough to suggest products as well as follow-up after a purchase is made. In some instances, customer service is being offered in real-time on the site. Some of the new online universities are also following this model. These universities are providing students with the means of obtaining a full education without ever stepping into a classroom. Students can now handle the entire process of their education from applying to the school, to class instruction, all the way to graduation from their home computer. Even counseling and course recommendations can be handled through an Internet connection. This does not appear to be the end of the evolutionary process. Already we have seen signs of another stage of growth. In this stage, all services are being incorporated into a single entity. Businesses are integrating the Internet into their business models so that there is no distinction between where the traditional business ends and the online begins. The lines between brick-and-mortar and electronic storefronts is disappearing at a rapid pace. So too can we expect the line between Internet based learning and class room learning do vanish. At some point the Internet will be so ingrained into our lives that there will no longer be the distinction of “online education”. Already we have seen this in the field of student research. In the past 5 years, the idea of “online research” being a separate entity from “general research” has vanished. No longer do we regularly hear professors and students commenting on the need to “go online” for research. It is generally assumed that some amount of research will be conducted online. This has already brought concern to some. Martin Hornyak, Brian Peach and Mick Fekula (2003) brought to our attention the possible problem with Web- logs, or BLOGS, and how students might have difficulty recognizing the difference to between good research and simple opinions, both of which are readily available and often undistinguishable on the web. Additionally, we have the difficulty of professors who are without experience or knowledge to effectively use these new technologies. Even with the new products being offered to handle the technical side of instruction, professors are still unprepared to effectively use these resources. Many universities have created offices or programs in hopes of instructing professors in how to use the technology as a pedagogical tool instead of simply as a medium of dissemination. With the huge push towards more integration of technology in the classroom, and little to no instruction on how to do so, there is little wonder why there is so much confusion on this topic. What technologies actually support and enhance the teaching environment? Which ones detract from it? Are there any that actually inspires the student to actively participate in their learning process? Or are they all dependent upon the desire of the student to learn and thus the technology has absolutely no bearing on the outcome? Obviously there are no clear answers at this time, or we wouldn’t be asking the questions. COURSE OVERVIEW It is for the purpose of bringing us closer to the answer of these questions that I present an evaluation of an Extended Distributed Learning course in which I participated during the summer of 2001. It fits the model previously described as it was taught as a distance education course with half the class located in Germany and the other half in the United States. These two classes were connected via two-way Interactive Television (ITV) and were taught by a professor from the U.S. while he was in Germany. To supplement the class work, the professor also required that the students participate in an online discussion forum. The discussion board was used as a place for students to post comments about their thoughts on a particular subject that the professor would initiate. Each student was expected to post a comment each week to further the discussion. They received marks pertaining to the quality of their comments and as to whether or not they added anything of value to the discussion. With a class of 30 U.S. students and 20 German, everyone was asked to keep their messages down to a reasonable size. During the 8 weeks of class, the professor tried to encourage the discussion in 3 separate ways. At first he provided a single topic and let the students comment at will for the week. He only added additional comments when asked a direct question. The second method was to ask 2 or 3 questions at the start of the week and then again, sit back and let the students discuss the questions without input unless a direct question was asked. The final method involved him actually taking a hand in directing the discussion by posing additional questions on the board when a topic appeared to be dying. 7 of the 8 weeks required individual 41 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 responses from the class members. There was a single week where group responses were allowed. Though annoying, this last minute mentality did not detract from the discussions. Where half of the class would wait to the last minute, the other half would participate at varying times during the week and actually provide a good solid discussion on the topic. With 50 students from two countries in the class, even half of the students provided a solid base of participants to further the discussion. And the most successful week appeared to be the ones where a lull in the conversation was supplemented with a quick summary and further questions (on the same topic) from the professor. The professor is an experienced instructor that has been teaching courses through ITV since 1998. In these instances, the courses were taught at the main campus and transmitted to a satellite campus about 45 minutes away. By alternating his presence between sites, he found that he could reduce the distress that the students at the satellite campus faced. This practice became common for ITV courses taught at the Business School. The distance education course taught from Germany was the first time this process of alternating could not be used. As such, he decided to use an online discussion to supplement the course. This was the first time that he had attempted to use a discussion board as a course requirement. When asked about this “last minute” behavior in class, the general response was that there was too much to read. Because it was all written comments, some comments would take up half to a full page of reading. The students were advised to check the discussion board daily, but not everyone was able to get on the Internet regularly. And those that did were then often faced with 5+ pages of material that they had to read at the end of a hard day at work. In order to keep up on the discussion, some students reported that they would check the board each morning and night. And then again in the afternoon if they had access. Additionally, students then had to take time to write a response at some point. Due to the asynchronous manner of this discussion, it was not always easy to keep on top of the discussion. In the time between a student reading the current comments and posting their reply, the discussion could have already moved on in another direction completely. This frustrated students who did not have the time to reply immediately. EVALUATION The evaluation was performed in two parts. The first was a general observation of the class itself, both in the formal classroom setting (from the U.S. classroom) and the online discussion. The second was a survey that was administered to the students at the end of the semester. The survey was designed as a tool to help in the future development of Distributed Learning courses and give feedback on direct technical issues as well as general teaching methods. As a general observation of the ITV from the U.S. side, the biggest issues appeared to be technical difficulties. It took some time for students to get used to the delay inherent in this technology. Anyone who has ever worked with this ITV before understands the 5-10 second delay in transmission. Though the transmission is nearly instantaneous, the computer needs 2-3 seconds to encode it, the receiving computer then needs another 2-3 seconds for decoding. Another session of delay is added if a response is required. If the student or professor was not patient, each could be 30 seconds into separate conversations before either side had been given a chance to respond. In this case, the professor was experienced in this delay and appeared to be able to handle the situation. The second concern is directly related to the first. In addition to adding a comment to the discussion, the students were expected to add something of relevance. There were many instances where a student would post their comment, only to learn that someone else had added the same idea just moment before. Other times, the students simply wanted to agree with what had already been said but due to the nature of the assignment, felt that they had to re-write the same comment in a different light in order to satisfy the requirements. Of course this exasperated our earlier concern of the reading load by adding even more to read, which would then turn-out to be a repetition of earlier comments. At one point, some students stopped reading the prior comments and simply posted their views and not deal with reading 10 postings that said the same thing. Of course this was an undesirable outcome as the idea was to have a discussion and not simply answer questions. The point of real interest came with the online discussion. The first week was a new experience for everyone and thus a full discussion did not take place. Some students seemed to understand the requirements, while others needed further explanation. But by the second week, the confusions had been ironed out. The real issues appeared to be in relation to the assignment directly. As stated before, each student was required to add something of significance to the discussion each week. There was a cut-off time each week when the comments had to be added. As a result, two main concerns arose In order to address the issue of “too much reading”, the professor switched things around a little bit by allowing the discussion to be handled in a group format. The idea behind this was that each group would work together to formulate thoughts that further the discussion. Groups would then post a single response for all 5 members and thus reduce the amount of reading that was necessary. Whether it was due to the time constraint of only one week, or another issue, that week’s discussion was little more than a single comment from each group on the week’s topic. Little effort was made to build on previous group comments and it appeared that once a comment was posted, many groups would not even follow-up with the discussion board for responses. The first area of difficulty is often called the “student syndrome”. Give a student a deadline and the assignment is accomplished as close to that deadline as possible. The online discussion was no different. Each week at least half of the students would wait until the final 24 hours before the deadline to add a comment. This last minute mentality made it difficult for a solid discussion to be held to flesh out ideas that the students were offering. For these students, the online board was (at least for that week) an “assignment” that had to be turned in and not a “class discussion” to discuss ideas. At the end of semester, the students were asked to fill out a questionnaire rating various aspects of the class and to freely 42 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 43 offer comments. The ratings that the students gave did not offer much information to evaluate the course, but the comments that they provided coincided with the general observations. There were complaints about technical difficulties with the ITV. There were the obligatory complaints about certain aspects of the opposite culture (though it was interesting to hear that the German students wanted more theory and the U.S. students wanted more application). There were a hand-full of comments from both classes that complained that the reading was excessive, though for different reasons. The German students felt that there was too much reading for the credits that they were receiving, while the U.S. students complained about having to juggle the large amounts of reading with work and their social life. But when asked to list the, “3 features about this class that you would improve”, there were only 3 students who asked for less reading. And on the flip side, when asked to list, “3 features you liked best about the class”, 30 out of the 50 students (20 U.S. and 10 German) listed the online discussion. And at least one of those students commented that they simply thrived on the discussion and was, “disappointed that it had to end”. The only part of the class that received more positive comments than the discussion board was the cross-cultural aspect. On the negative side, only technical problems received more comments than the 3 complaining about the reading load. SUMMARY & SUGGESTIONS Though not conclusive, the comments and general observations do lead me to believe that the online discussion was a positive experience. Though not discussed, the experiential nature of the online discussion alone promises to enhance teaching efforts. Students that are intimidated or need time to formulate responses could go online and actively participate in their learning. The learning would not be limited to the time frame of the actual class session. For future use of the online discussion board, the students provided the following suggestions. By implementing some of these strategies, we will be able to increase the quality of the use of discussion boards as a learning tool. • Provide a “me too” option that would allow students to agree with a particular comment without having to repeat the idea. This though would have to be tempered in such a way that it didn’t become an easy way out of participating in the discussion. • Spread the topics over a longer length of time. Instead of a single week per topic, it could be spread out to 2 weeks or longer. If done as a rolling topic (ie. no set date when a topic will begin or end), the instructor could lengthen or shorten the discussion depending on the level of student involvement • Do not require a weekly submission by the students. By requiring the students to participate in 4 out of 8 weeks discussions, they would be given the option to choose topics on which they could provide positive commentaries. This would also reduce the repetition of ideas and thus reduce the amount of reading. • Make the online section a “participation” grade instead of an assignment. Most professors have some form of participation grade that does not require a certain number of comments from their students while in the classroom. Extend this participation grade to the online arena. This would, in conjunction with previous suggestions, remove some of the “student syndrome” problems. REFERENCES Burns, Alvin (1998). A Neophyte Distance Educator’s Experience; Development in Business Simulation and Experiential Learning, volume 25, 138-144. Reprinted in The Bernie Keys Library, Hugh M. Cannon (ed). [Available from http://www.ABSEL.org] Cates, Tommy, Rayburn, Michael and Green, Robbie (1996). Interactive Distance Learning as a Tool in a College’s Theory and Practice; Development in Business Simulation and Experiential Learning, volume 23, 160-161. Reprinted in The Bernie Keys Library, Hugh M. Cannon (ed). [Available from http://www.ABSEL.org] Frank, Sarah (2000). Collaborative Learning and Web-Based Instruction in a Cognitive Apprenticeship Model; Development in Business Simulation and Experiential Learning, volume 27, 188-194. Reprinted in The Bernie Keys Library, Hugh M. Cannon (ed). [Available from http://www.ABSEL.org] Gold, Steven (2001). E-Learning: The Next Wave of Experiential Learning; Development in Business Simulation and Experiential Learning, volume 28, 76-79. Reprinted in The Bernie Keys Library, Hugh M. Cannon (ed). [Available from http://www.ABSEL.org] Gosen, Jerry (2003). A Model for Online Education Delivery and a Look at Online Delivery Effectiveness; Development in Business Simulation and Experiential Learning, volume 30, 279-287. Reprinted in The Bernie Keys Library, Hugh M. Cannon (ed). [Available from http://www.ABSEL.org] Hara, Noriko and Kling, Rob (2000). Students’ Distress with a Web-based Distance Education Course: An Ethnographic Study of Participants’ Experiences; Information, Communication & Society, 3 (4), 557-559. Also available online at www.slis.indiana.edu/csi/wp/wp00-01b.html Pillutla, Sharma (2000). Creating a Comprehensive Web- Enhanced Classroom; Development in Business Simulation and Experiential Learning, volume 27, 210-216. Reprinted in The Bernie Keys Library, Hugh M. Cannon (ed). [Available from http://www.ABSEL.org ] Potosky, Denise (2002). Virtually Experiential Classrooms; Development in Business Simulation and Experiential Learning, volume 29, 172-178. Reprinted in The Bernie Keys Library, Hugh M. Cannon (ed). [Available from http://www.ABSEL.org ] http://www.absel.org/ http://www.absel.org/ http://www.absel.org/ http://www.slis.indiana.edu/csi/wp/wp00-01b.html http://www.absel.org/ http://www.absel.org/ Table of Contents Volume 31, 2004 Controlling the Complexity and Orenting Target Groups by a Modular, Server-Based Business Game System Learning Network Demonstration: Delivering Business Education in a Distance Learning Environment Economic Evolution, Human Capital Investment, and Adult Distributed Electronic Learning: A Literature Review Designing a Globalization Simulation to Teach Corporate Social Responsibility Developing and Teaching an Online / In-Class Hybrid: A Demonstration A Model for Evaluating Online Instruction An Evaluation of a Distributed Learning Course: A Students'-Eye Perspective Blended Learning Strategy Improved Business Writing Skills How to Receive and Process Attachemnts while Greatly Reducing the Risk of Viruses and Trojans Introducing Online Components to a Class: How to Increase teh Likelihood of Success Teaching Strategic Communications Online: Using Learning Outcomes to Develop a Case-Based Course Implementing Distance Approaches to Education: A Panel Discussion for ABSEL: Las Vegas, 2004 MANDI: Learning Management Through Field Sales Experience An International Capital budgeting Experiential Exercise A Primer To Combating Terrorism: Playing It Safe While On Overseas Assignment (An Experiential Exercise) Integrating The Business Curriculum With A Comprehensive Case Study: A Prototype The Case Brief: A Model For Case Analysis, Writing And Discussion Technology Infused Pedagogy And Delivery – A Sure Bet? A Proposal For Panel Discussion Absel Conference 2004 Research Strategy And The Bkl: Getting The Most From The Absel Archives Simple But Effective: Rediscovering The Class Discussion Needle And Thread: An Activity For Examining Various Management Behaviors A Customized Excel Data Analysis System For Use In Undergraduate Marketing Research Team Leader Selection - Does It Matter? The Power Of Perspective: Reframing Your Framing Skills For Innovative Instruction In Leadership And Influence Exercise: How Should Merit Raises Be Allocated? An Online Situation For Problem-Based Learning In A Junior-Level Management Course The Eden Alternative As A Roadway For Change: A Service Learning Quality Improvement Project Avoiding Catastrophe: The Role Of Individual Accountability In Team Effectiveness Omega Systems: A Change Management Exercise The Risks And Rewards Of Providing Students A Structured Cheating Opportunity Experimentation With Assessment Techniques: A Proposal For Panel Discussion Using A 2 - Page Case To Introduce Concepts Of Business Strategy Interactive Session The Integration Of Appreciative Inquiry And Experiential Learning For Peak Performance Appreciative Inquiry Case Story: New York City Leadership Challenge Individual Achievement Versus Team Performance: An Empirical Study With Business Games Some Strategists Don't Learn Or Can't Learn Computer Simulation: A Design Architectonic On The Value Of Bugs In Simulation Environments Online Sales Forecasting With The Multiple Regression Analysis Data Matrices Package Simulation Exercises And Problem Based Learning: Is There A Fit? A Study Of Business Game Stock Price Algorithms Assessing Individual Performance In A Total Enterprise Simulation Information Use In A Business Game Determining The Value Of A Firm Unsorting Algorithms For An Ordered List And Its Application To Business Simulations Teaching Public Finance Management Through Simulation Antecedents Of Game Performance Student Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Implementation And Impacts Of The Balanced Scorecard: An Experiment With Business Games Impact: Shocking The Legacy Mindset Implementation Of The Eepad Framework Of Business Processes In An Accounting Information Systems Course Are Business Games Really Delivering What Students Are Led To Believe?? Reporting Lessons Learned: What Gets Reported; Who Gains Value Teacher Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Student Reactions To The Use Of A Computer-Based Simulation As An Integrating Mechanism For A Mba Curriculum A Cognitive Investigation Of The Internal Validity Of A Management Strategy Simulation Game The Casino Challenge: Making Simulation Delivery A Safe Bet! Accounting For Company Reputation: Variations On The Gold Standard Foreign Currency Hedging: A Simulation The Influence Of Variables Easily Controlled By The Instructor/Administrator On Simulation Outcomes: In Particular, The Variable, Reflection. Absel Awareness Among Business School Faculty Validating Business Simulations: Does High Market Share Lead To High Profitability? Simulation Debriefing Procedures Coaching And Business Simulations: A Formula For Success? A Seminal Inventory Of Basic Research Using Business Simulation Games The Influence Of Scorecard Evaluation On Decisions And Outcomes