INAPPROPRIATE USE OF CITATIONS AND CORRUPTING THE BODY OF KNOWLEDGE: ACCEPTING URBAN LEGENDS AS TRUTH Developments in Business Simulation and Experiential Learning, Volume 34, 2007 314 INAPPROPRIATE USE OF CITATIONS AND CORRUPTING THE BODY OF KNOWLEDGE: ACCEPTING URBAN LEGENDS AS TRUTH Richard G. Platt The University of West Florida rplatt@uwf.edu E. Brian Peach The University of West Florida bpeach@uwf.edu ABSTRACT Peach & Platt (2002) contended that one of the critical aspects of any good research publication is the list of references provided by the authors. Such a list provides readers not only with insights into the rigor of the current effort, but an immediate and convenient pathway to the relevant previously published research upon which the current research is based.. Even occasional failure to properly cite references can allow false or misleading information to be incorporated into what could be considered ‘accepted truth’ that many people might feel does not need a reference. This paper provides examples of supposed research widely accepted as truth, one with significant potential impacts on education and the choice of pedagogical approach. It concludes with recommendations for maintaining rigor in the citation process. Keywords: Research, Citations, Dale’s Cone INTRODUCTION The purpose of this paper is to build on the efforts of Peach & Platt(2002) who found that some conferences did not include references in their proceedings due to page limitation constraints based upon logistical and cost limitations associated with the production of hard copy, printed proceedings. In addition, they found there was a “failure by a significant group of authors to provide any cites for their works” (2002, p260). Their focus was on the limits this omission placed on the reader’s ability to consult the source of the authors’ references or conclusions. This paper builds on their work and addresses the equally critical problem that inaccurate or inappropriate citations can lead to inferences by readers that the citations or data included in a paper are based on rigorous peer-reviewed research when in fact they may have started out as opinion and transmogrified into ‘truth’ through faulty citations. This has potentially grave consequences for those who rely on peer reviewed research to contain rigorous reporting. Academia must be able to rely on the peer review process. Academic research has long been founded on the principles of building on and extending the body of knowledge. Thus the foundation of any research project is the body of work that precedes it as published in the relevant literature. Assessing the validity and value of any new work is predicated not only on knowing what has already been done, but also on knowing the validity of what has already been done. Only by knowing what has already been done in the field can either the author or reader effectively ascertain the benefit and relevance of the work at hand. If authors do not perform a thorough and accurate literature review, it is difficult for them to know how their contribution fits into the research stream of previous efforts. To the extent that references have errors and/or are not provided to readers, a number of negative issues are raised. First, it is difficult for uninitiated writers to accurately ascertain the value and level of contribution their paper makes. Second, it impairs the ability of other researchers to refer to source documents and to build on the paper’s contribution. Third, if a citation inappropriately identifies information as originating from peer reviewed material, it falsely provides the implication of legitimacy to misleading or even patently false information. In today’s world, academics as well as others are increasingly concerned about the nature and volume of false information available on the Internet and transmitted through e-mails. Students and the general public often accept as true material that either has strong face validity or information that they want to believe. The situation is aggravated by the fact that most Internet users are not trained to look for sources and references and no standards exist for documenting sources (Graham and Metaxis, 2003). Thus, the Internet provides a frightening example of what academia’s published research could look like if rigor is not maintained through consistent provision of accurate and complete citations. The educational system at both undergraduate and graduate levels emphasizes the need to cite references for sources used in preparing reports and papers. However, the emphasis on citations in the collegiate educational system appears to be focused on preventing plagiarism, as well as explicating the academic role citations accomplish (e.g., Hunter, 2006; UNCSW, 2006). mailto:rplatt@uwf.edu mailto:bpeach@uwf.edu Developments in Business Simulation and Experiential Learning, Volume 34, 2007 315 INAPPROPRIATE CITATIONS The term ‘inappropriate citation’ covers a wide range of possible actions. First, an author could discuss a topic that was clearly derived from an outside source but fail to provide any citation at all. Although this may be due to laziness or lack of time or easy access to the source, it often occurs because the author believes ‘everyone knows’ both the source and validity of the information. Regardless, it devalues the peer review process. A second category we place under the rubric ‘inappropriate citation’ is when the publisher of the article fails to include the list of references. The major source of this historically was proceedings that for reasons of economy and space did not include references, opting for the ‘references available upon request.’ Such an approach seriously impedes researchers for two reasons. First, they would have to spend considerable extra effort to verify perhaps only one or two references. Second, as the article ages, it may get progressively more difficult to locate the authors, negating any possibility of obtaining a proper list of references. A third category of inappropriate citation occurs when a citation has errors in it that either limit or destroy its usefulness: it may have the wrong author, date, or volume, or other errors that corrupt the identification of the true source. A fourth category covers the tendency of some authors to directly cite references that were cited in a source article, instead of using the proper “as cited in” format. Thus, if the source article had an incorrect citation, this incorrect citation can promulgate across additional articles in their list of references. A fifth category is where a reference is cited but the cited work does not actually contain the relevant material. Whether this occurs because the author was operating on memory as to where the material was obtained, or there were errors in research notes, the net effect is the same. The chain of reference is broken. The sixth, and perhaps most insidious category, is when an author adds to or otherwise modifies cited material, but the citation structure implies that all of the discussion was in the cited material. Even a careful researcher that verified sources might not pick up on a small change in emphasis or meaning without a very careful reading of the source. Researchers depend on accurate summaries and representations of cited works. Because the role of citations in academic research is so critical, the vast majority of researchers are diligent in accurately and completely referencing their research. The purpose of this paper is not to contend that a rampant problem exists in the quality of citations. The authors do contend that the quality of citations is a precious commodity that cannot be squandered through less than rigorous maintenance of editorial standards. The Dale’s Cone transmogrification is an example of what can happen when rigor slips. URBAN LEGENDS We now live in an era where the Internet has become the first source for information for many people, but in the process has spawned a disinformation dissemination process which inextricably interweaves fact, fiction and fantasy until even sophisticated readers may not be able to discern the validity of Internet-based communications. Such disinformation is known as an urban legend. Although there is no single universally accepted definition for an urban legend, most of the definitions share a degree of commonality in the definition. The use of the term ‘urban’ confuses many as urban legends are not limited to urban settings. Wikipedia states that urban legends are a form of modern folklore and thus the term urban separates urban legends from classic folklore (Wikipedia, 2006). The legend part of urban legend derives from the nature of urban legends in that although they may not always have the story- like elements of classic legends, they frequently take the form of short stories. For the purposes of this paper, “an urban legend is any modern, fictional story, told as truth, that reaches a wide audience by being passed from person to person” (How Stuff Works, 2006). Other areas of commonality among the many definitions of the term urban legend include that the legend is often something that readers might wish were true, that the legend appeals to the readers’ interests, that the legend spreads quickly, and finally that the legends take on a life of their own, so much so that they appear to become truth by the very nature of being repeated so often. Although urban legends are typically false, they are often based on an actual person or event. Thus the urban legend can be totally false, partially false, or a distortion of events, but retain a seemingly reasonable link to reality as readers perceive it. Because of the nature of how urban legends are created and disseminated, “More often than not, it isn't possible to trace an urban legend back to its original source -- they seem to come from nowhere .” (HowStuffWorks, 2006) Performing a quick Google search (http://www.google.com) for the term “urban legend”generates over 2,620,000 results for the phrase. Anyone who has used email for only a few months has run across at least one email containing an urban legend. It might have been the Bill Gates-Microsoft-AOL giveaway (http://www.snopes.com/inboxer/nothing/microsoft- aol.asp), or it might have been the Andy Rooney monologue (http://www.snopes.com/politics/soapbox/rooney4.asp), or even the flashing headlights-gang initiation (http://www.snopes.com/horrors/madmen/lightsout.asp). These three as well as the other 2,620,000 are all examples of urban legends, are all false, and were all passed from one person to another as if they were truth. Although this paper cited a Wikipedia definition, Wikis also spawn their own form of urban legend. The very nature of the technology used to create a Wiki means that a Wiki is a form of open-source (Intelligence Terms and Definitions, Developments in Business Simulation and Experiential Learning, Volume 34, 2007 316 2006) repository that relies on its member community to police the accuracy of the information found on the Wiki’s web pages, not editors or reviewers who are knowledgeable in the field, but just members who believe they are well informed. Sometimes postings on a Wiki creates a situation of the blind leading the blind, or the prejudiced leading the prejudiced, or the perpetrating of a hoax just as in an urban legend. An example of this was the infamous character assassination attack on John Seigenthaler Sr. Someone posted a hoax biography on a wikipedia that was later proven to be false, but only after being posted and viewed for over four months. (http://www.cbsnews.com/blogs/2005/12/07/publiceye/entry 1103143.shtml) The quality and veracity of a wiki must be treated as highly suspect. One participant in a discussion of this issue posited that “Whoever (sic) quotes Wikipedia as a source in an argument automatically loses the argument..” (http://www.rogerlsimon.com/mt- archives/2005/12/_wikipedia_use.php). CREATING A RESEARCH URBAN LEGEND Peer reviewed research, by the very nature of the peer review process, could never spawn its own urban legends. Or could it? Witness the following example. At a recent international conference, a reputable authority from the teaching center of a university classified as a Research I institution presented a program on how to improve teaching in an online environment. One of the artifacts used in the presentation was the chart found in Figure 1. The values found for the categories in Figure 1 supported the themes of the presentation and were well-received by the audience. After all, the material was presented by an authority in the field and supported by a reference citation, which purported to substantiate the validity of the categories and values found in Figure 1. Several months after the conference and after referring to the content from Figure 1 in other venues, the authors discovered a web site that purported to debunk the validity and authenticity of the material found in Figure 1 (Work- Learning Research). Reluctant to give up on an artifact that was both appealing and apparently plausible, the authors were not about to take another author’s word for something without verifying the sources. A quick trip to the library located a copy the book cited as the source of the material in Figure 1 (Uno, 1999). Although the content from Figure 1 was found verbatim in Uno’s Handbook, there was no accompanying reference citation indicating the source from which the material was taken. A quick email to the author asking the source resulted in a pointer to the probable source ( Uno, 2006). And the chase was on: what was the original source for this model and what was the quality of the research that generated the percentages as found in Figure 1? The Cone of Experience was first introduced in Edgar Dale’s (1946) book on Audiovisual Methods in Teaching. Dale subsequently updated his Cone of Experience in later editions of Audiovisual Methods in Teaching published in 1954 and again in 1969. The version of Dale’s Cone of Experience found in the 1969 edition of his book is found in Figure 2. Notice the similarity between the categories in Figure 1 and the categories in Figure 2. However, the numeric values found in Figure 1 do not appear in any of Dale’s three editions of Audiovisual Methods in Teaching. One source often cited for Dale’s Cone is Educational Media: Theory into Practice edited by Wiman and Meierhenry (1969). The problem is that none of the edited chapters contains the diagram with the percentages. So, that source is not valid either. What is even more telling is that there are 19 references found using a Google search that use the same misspelling of Meierhenry’s name, spelled as “Mierhenry”, in conjunction with the reference to Dale’s Cone of Experience. Others (eight in a quick Google search) spell Wiman’s name as Winman to the same effect. Figure 2: Dale’s Cone of ExperienceThalheimer Figure 1 HOW WE LEARN 10% OF WHAT WE READ 20% OF WHAT WE HEAR 30% OF WHAT WE SEE 50% OF WHAT WE SEE & HEAR 60% OF WHAT WE WRITE 70% OF WHAT WE DISCUSS 80% OF WHAT WE EXPERIENCE AND 90% OF WHAT WE TEACH (each other) Adapted from "Handbook on Teaching Undergraduate Science Courses" by Gordon Uno By Permission Developments in Business Simulation and Experiential Learning, Volume 34, 2007 317 (2006) attributes the numbers found superimposed onto Dale’s Cone of Experience to work published by D.G. Treichler in 1967 in the magazine Film and Audio-Visual Communications. This article provided the percentages but without any supporting research. Others attribute the Cone of Experience to National Training Labs in Bethel, Maine (Qayumi, 2006) which does not have a single peer-reviewed publication showing how their version of the cone experience was validated. Regardless of which version of the Cone of Experience you choose, what remains is an idea that appeals to the readers’ interests, an idea that has spread over a period of years, until finally the idea takes on a life of its own, so much so that they appear to become truth by the very nature of being repeated so often. That is the definition provided above for an urban legend. RECOMMENDATIONS Urban legends can arise from a variety of sources spreading disinformation to a vast number of people seemingly almost overnight. Hopefully, although this may cause individual perceptions to be incorrect and beliefs unfounded, the impact is not fatal for society. Within academia, the inappropriate use of citations is far more unusual and the spread is much slower. The impact, however, may be more serious. Resources can be wasted pursuing flawed or misrepresented research. As in the case of Dale’s Cone, educators can base their actions and pedagogical approaches on non-existent research. Even more serious impacts can be easily envisioned. We can deal with Internet based urban legends by not accepting something as true because it was passed on by a trusted friend and consulting snopes.com when suspicious material reaches our attention. Maintaining the rigor of the academic peer review process requires more systematic and careful attention. Publishing outlets such as journals and conference proceedings must continually reinforce the importance of accurate and complete citations through not just their published standards, but also through assiduous enforcement of their editorial review policies. Although it is incumbent upon all authors to maintain high standards in citing material, the rigor of good citation procedures can only be maintained through a careful and professional review process. Top quality journals are known for good review practices including a strong emphasis on quality of references. A sure and quick way to initiate a deteriorating quality of submissions and acceptances is to allow poorly referenced papers into the publication cycle. All academics, from editors through reviewers to authors, must contribute to the continuing integrity of academic research by ensuring they place a strong emphasis on proper citation procedures. But the final check lies in an effective peer review process which must include confirmation of the veracity of works cited and the accuracy of the reference citations. REFERENCES The Bernie Keys Library (version 2) [CD-ROM]. 2001. The Association for Business Simulation and Experiential Learning [Producer and Distributor]. Dale, Edgar. (1969) Audiovisual Methods in Teaching, 3rd ed. New York, Dryden Press. Graham, L. and P. T. Metaxas. 2003. "Of course it's true; I saw it on the Internet!": Critical Thinking in the Internet Era. Communications of the ACM 46, 5 (May. 2003), 70-75. How Stuff Works. Urban Legend. Retrieved 10/28/2006 from http://www.howstuffworks.com/urban-legend.htm Hunter, J. 2006. The importance of citation. Hunter College. Retrieved 10/18/2006 from http://web.grinnell.edu/dean/tutorial/EUS/IC.pdf Intelligence Terms and Definitions. Open Source. United States Intelligence Community. Retrieved 10/30/2006 from http://www.intelligence.gov/0-glossary.shtml#5. Peach, E. B. and R. G. Platt. 2002. The ABSEL Research Heritage and the BKL: Leveraging Their Value For Future Research. . Developments in Simulation and Experiential Learning, 29, 260-264. Platt, R. G., and E. B. Peach. 2001. The ABSEL EndNote database: The Perfect Tool for the Bernie Keys Library. Developments in Simulation and Experiential Learning, 28, 201-204. Qayumi, Karim. Centers of Excellence: A New Dimension in Surgical Education. Surgical Innovation, 13,2 (June 2006), 120-128. Uno, Gordon E. Handbook on teaching undergraduate science courses : a survival training manual. Fort Worth : Saunders College Publishing, 1999. Uno, Gordon E. Personal communication. September 26, 2006. UNSW. 2006. Why is referencing important? University of New South Wales. Retrieved 10/18/2006 from http://www.lc.unsw.edu.au/plagiarism/citation.html Walstrom, K. A., and L. N. K. Leonard. 2000. Citation classics from the information systems literature. Information & Management, 38, 59-72. Wikipedia. Urban Legend. Retrieved 10/28/2006 from http://en.wikipedia.org/wiki/Urban_legend. Wiman, Raymond V. and Wesley C. Meierhenry. (1969) Educational media: theory into practice. Columbus, Ohio, Merrill. Thalheimer, Will. People remember 10%, 20%...Oh Really?. Work-Learning Research. Retrieved 9/26/2006 from http://www.work-learning.com/chigraph.htm http://web.grinnell.edu/dean/tutorial/EUS/IC.pdf http://www.intelligence.gov/0-glossary.shtml#5. http://www.lc.unsw.edu.au/plagiarism/citation.html http://en.wikipedia.org/wiki/Urban_legend. Table of Contents Volume 34, 2007 Experiential Teaching May Lead To Experiential Learning The Role Of Learning Versus Performance Orientations When Reacting To Negative Outcomes In Simulation Games: Further Insights An Analysis Of The Interaction Of Firm Demand And Industry Demand In Business Simulations Consistency Of Participant Simulation Performance Across Simulation Games Of Growing Complexity Assessing And Incentivizing Learner Contribution And Performance Excellence: Creating Win-Win Evaluation Options The Technological Impact Analysis: A Research-Based Exercise To Heighten Learners' Technological Sensitivity Beginnings: How We Start the Semester and Individual Classes : A Roundtable Discussion Simulation Performance and Its Effectiveness As A PBL Problem: A Follow-Up Study The Use of Multimedia Learning Tools to Facilitate Online Learning of Business Statistics Forecasting Accuracy And Learning: A Key To Measuring Business Game Performance Panel Discussion: Alternative Ways of Using the Internet for Business Simulations to Input Decisions, Process, and Present Financial and Economic Data Output Absel Research -- One Additional Perspective On Where We Are And Where We Have Come From Assessing participant learning in a business simulation From Case Presentation To Case Facilitation: How Assessment Changed The Capstone Course The Use Of Computer-Assisted, Interactive Role-Play Simulation In Hong Kong Corporate Positioning: A Business Game Perspective Demonstration Of A Computer-Assisted Global Business Simulation Applying .NET Remoting To A Business Simulation Assessing Emotional Intelligence: The EQ Matrix Exercise Outcomes And Observations Of An Extended Accounting Board Game The Use Of Learning Styles Questionnaire In Hong Kong Forming Teams For Classroom Projects Online Budgeting And Marketing Control With The Proforma Analysis Package The Effect Of Experiential Learning Experiences On Management Skills Acquisition Students' Perceptions on the Individual Managerial Performance Orderness: A New Definition Of Alignment An Exploratory Study Of The Efficacy Of Servsafe® Online Team Behavior And Team Success Results From A Board Game Simulation Telecommuting Internships - Do They Work? The WEE GAME: A Pre-Game Effects On Learning When Students Have Information About Games And Their Outcomes When Playing Them Assessment And Simulations: Measuring The Academic Learning Compacts Within Dangerous Business: An Interactive Ethics Case Activity The Paradise Islands Assessing And Applying fiAppropriatefl Conflict Management Styles Distance Learning In Communication: Blended Or On-Line? Developing An On-Line Advanced Communication Course World Café: Simulating Seminar Dialogues in a Large Class The Programming Game: An Exploratory Collaboration Between Business Simulation And Instructional Design The Canary Principle: An Alternative Model for Providing Real-Time Coaching in an On-Line Discussion Environment Assessing Character Development Using A Direct-Approach Business Ethics Exercise Persistence Of Decisions By Simulation Game Participants The Invalidity of Profit = f(Product Quality) PIMS Validation of Marketing Games A Study of The Applicability of The Perceptions of Organizational Politics Scale (POPS) For Use in The University Classroom The ROI Of Connected Projects: A Payroll Example Initiation of Research on Gaming Simulation in Japan Inappropriate Use of Citations and Corrupting the Body of Knowledge: Accepting Urban Legends as Truth An Exploration Of The Perceived Value Of Highly Socio-Inductive Learning How "Whole" Is Whole Person Learning? An Examination Of Spirituality In Experiential Learning Modeling Outsourcing and Strategy Alignment into a Business Game Successful Integration Of Webct Into A Small Business School The Thinking Steps Model In Game Theory: A Qualitative Approach In Following The Rules The Application Of Means-End Theory To Understanding The Value Of Simulation-Based Learning Armchair Travel: An active learning approach to increasing global awareness and participant self-efficacy Goldratt's Thinking Process: Is there a place for it in the Total Enterprise Simulation Assessing And Developing Student Skills Using a Group Exercise Computer Business Simulation Design: Novelty & Complexity Issues