Research Symposium Report Journal of Learning Development in Higher Education ISSN: 1759-667X Issue 2: February 2010 Using Turnitin as a formative writing tool Judy Cohen University of Kent, UK Abstract Turnitin plagiarism detection software was made available to all staff at the University of Kent at the beginning of the 2006/7 academic year. To promote the potential positive aspects of Turnitin, the Unit for the Enhancement of Learning and Teaching (UELT) prepared guidelines for staff on how to use Turnitin formatively rather than as a plagiarism policing agent. In addition, extensive web resources on academic integrity and avoiding plagiarism were made available to staff and students. It was anticipated that the guidelines and web resources coupled with tutor support in class would facilitate student understandings of plagiarism, referencing and the development of writing skills. Turnitin was specifically trialled in two classes of students, with interviews of staff and students on their experience of Turnitin and how it influenced their academic writing. Staff concluded that Turnitin and specific in-class support significantly deterred plagiarism, while students supported the use of Turnitin and felt that it helped them understand the actual process of integrating references into their own work. Context While plagiarism itself is not a new phenomenon, there is increasing concern in academic circles that plagiarism, particularly from Internet sources, is on the rise (Chester, 2001; McDowell and Brown, 2001; Johnston, 2003). This perceived rise in plagiarism in recent years has led to a substantial body of work concerned with defining, detecting and deterring plagiarism. A generally accepted principle in this work is that plagiarism should be tackled holistically, incorporating institutional policy, practice and culture; assessment design; Cohen Using Turnitin as a formative writing tool education of and information for students; detection; and finally, appropriate application of penalties (Carroll, 2002). While some of the blame for plagiarism is placed on the use of the Internet and the corresponding rise in the ease with which students can access information online (Chester, 2001), others have pointed out that changes to higher education practices are also significant. These changes include the increased access and ‘massification’ of higher education; increased use of coursework in assessment and the importance of grades and increased competition (McDowell and Brown, 2001). Regardless of the root causes of either plagiarism or the worry over the increased ease that students can access ‘copiable’ material, the first step in dealing with this form of academic misconduct is to establish a common understanding of plagiarism (Carroll, 2002; Johnston, 2003). As Johnston (2003) points out, improving students’ understanding of what is and is not acceptable practice when using sources is more beneficial than focusing on detection and penalties. However, he adds the caveat that developing a common understanding of plagiarism is complex in itself. In addition, a recent case study shows that developing an understanding of plagiarism does not necessarily mean that students can meet academic writing requirements of avoiding plagiarism in written work. Instead, this understanding of plagiarism should be coupled with detailed formative workshops on developing these particular academic writing skills (Sivasubramaniam, 2006). It is this aspect of linking the understanding of plagiarism in the students’ minds with the writing skills necessary to avoid plagiarism that concerns this case study and how Turnitin was used to facilitate this process at the University of Kent. Turnitin was acquired in 2006 against the backdrop of concerns regarding plagiarism in the HE sector, after a process of overhauling plagiarism-related policy documents and piloting the text matching software in 2005. The University adopted an educative and formative approach to the use of Turnitin, including the holistic approach to deterring and detecting plagiarism as advocated by Carroll (2002) and in line with Peacock and Sharp (2006). Guidelines on using Turnitin (http://www.kent.ac.uk/uelt/ai/students/guidelines.html)were developed and approved at the start of the academic year, and later piloted in two modules: Economics (level 2) and Journal of Learning Development in Higher Education, Issue 2: February 2010 2 http://www.kent.ac.uk/uelt/ai/students/guidelines.html Cohen Using Turnitin as a formative writing tool Business (Level 3). This case study explores the staff and student experiences of using Turnitin as a formative writing tool in these two modules. Research questions Initially this study set out to investigate student understandings of: • What Turnitin is and its implications for them; • How to interpret the originality report; and • Using the originality report to improve their understanding of academic writing conventions. This study also aimed to informally discuss with staff the impact of using Turnitin: • On incidences of plagiarism within the module; and • On workload issues in investigating cases of plagiarism. Brief methodology Surveys of staff along with case studies (including student surveys and interviews) of two modules were carried out to explore experiences of implementing Turnitin in departments and specifically in using Turnitin to develop academic writing skills: • Staff were surveyed to ascertain their initial understanding of Turnitin Plagiarism Detection software; the university’s approved guidelines on using Turnitin; and their expectations for the software. For example, questions included ‘I am concerned that using Turnitin will increase my workload’ and ‘I believe that using Turnitin will be a useful aid in deterring plagiarism’. Surveys also included departmental questions on existing information for students on plagiarism. Journal of Learning Development in Higher Education, Issue 2: February 2010 3 Cohen Using Turnitin as a formative writing tool • Students were given two introductory sessions (one information and one practice session) about the software including how to log in to Turnitin and upload work followed by how to interpret the originality report. The practice session on the originality report provided a chance for the students to discuss what the matching percentages meant and to see the extent to which their writing matched other source material. Students were surveyed after the Turnitin information and practice sessions and interviewed at the end of their module. The Business group (41 third year undergraduate students) were allowed to upload a previously assessed piece of work and were allowed free access to Turnitin, with the understanding that this was formative practice and that the work would not be re-examined or assessed regardless of the originality report results. Referencing and citation advice was provided and the subsequent assessed work was later submitted once only with no opportunity to view the originality report (these settings were fixed in the Turnitin assignment options). When submitting work to Turnitin, tutors have the option to allow students to see their originality report or not, to allow single or multiple submissions (meaning students can edit their work and resubmit to Turnitin), or to create draft/revision assignments where a single draft and final copy of the submission and the originality report are saved. The originality report is a colour coded version of the submission highlighting text which matches other sources, whether these are adequately referenced or not. In the Economics module, the 35 level 2 students were allowed to upload any piece of work to Turnitin in the first session with free access to Turnitin provided for a limited time (around one week). Their final assessed piece of work was submitted by a draft/revision assignment. In this format of assignment, the tutor has a copy of the draft essay and associated originality report plus the final submission and report. Students can submit the draft and final copies once only. Journal of Learning Development in Higher Education, Issue 2: February 2010 4 Cohen Using Turnitin as a formative writing tool Discussion of findings Staff During the initial stages of implementing Turnitin, a number of concerns were raised by staff and students with the Turnitin Administrator at the university. Staff concerns included: • Practicality of electronic submission: after initial training, most staff and students found that this was a relatively straightforward process; • Printing assignments or how to match printed and electronic submissions: this remained a concern, particularly when it was realised that Turnitin occasionally created an error between an uploaded Word document and the associated image used by GradeMark. In terms of cross-checking print versions and Turnitin reports, staff relied on random checks; • The validity of using Turnitin as a sampling tool (random checks on individual students): this approach was not in line with the University guidelines on using Turnitin and was not an accurate gauge of the standard of work for a class as a whole. Students were concerned about being singled out; • Confusion over the meaning of the originality report percentage: this was a concern to staff who were tempted to just check high percentage matches without considering the implications of plagiarism occurring with lower percentage matches. Students were concerned about the fair reading of their work or the risk of being judged by a number on a report; • Support when pursuing cases of plagiarism: staff imagined that if they found and reported more cases of plagiarism, they may not get administrative support while involved in dealing with the formal processes of each case. Staff response to surveys indicated that although staff were generally keen to use Turnitin, they were unsure how it may impact on their workload and also unsure of the University guidelines on implementing Turnitin. Interestingly, one staff member during interview reported a significant drop in the time taken to follow up cases of plagiarism as Turnitin provided adequate evidence and further time spent searching for plagiarised sources was not required. In this way, Turnitin had a positive effect on workload. It was also reported that Turnitin reduced time spent marking, either for the reason above, or due to a significant drop in Journal of Learning Development in Higher Education, Issue 2: February 2010 5 Cohen Using Turnitin as a formative writing tool instances of plagiarism. Of the departments that responded to the survey, all had a definition of plagiarism available, and of those intending to implement Turnitin, the anticipated benefits included detecting plagiarism and changing the student attitude towards cheating. One department noted that the use of Turnitin had had the unexpected effect of dramatically increasing the amount of direct quotations used by students. Students Prior to receiving information on Turnitin, students were anxious about using the system, particularly in terms of the fair application of the software and the risk of being unjustly accused of plagiarism (anecdotal evidence reported in conversations between the Turnitin Administrator and Student Union representatives). Student concerns included: • Insecurity about how the software worked: including the fear of being unjustly accused of plagiarism or of staff believing the originality report and not the student; • The possibility of the introduction of bias in marking if staff rely on the originality reports: more concerns about being treated as a number on a report and losing the personal aspect of a staff member marking their work, with concerns about bias if a report showed a certain matching percentage; • The setting of acceptable minimum/maximum percentages for matching text: students rightly were concerned about this as it is a meaningless exercise; and • The possibility of staff using the system without adequate training or understanding (raising concerns over fair treatment of students): anecdotal evidence from a range of students across the campuses showed that staff from different departments did not always have the same level of expertise with either the software or the University guidelines; • Ownership of intellectual property rights when work is submitted to the Turnitin database (a concern shared by staff): a concern not really affected by Turnitin. Students at the university can request to have any confidential material (that falls under Data Protection rules) removed from the Turnitin database, and any work with Journal of Learning Development in Higher Education, Issue 2: February 2010 6 Cohen Using Turnitin as a formative writing tool commercial potential is either covered by current university rules on intellectual property, or, if part of higher degree research, can also be removed from the database. Student survey results After the information sessions on Turnitin, 85% of the Business students reported that they felt they had received enough information about Turnitin (strongly agreed or agreed) while 90% were confident that they understood why Turnitin was being used. In the Economics group, 94% felt they had received enough information about Turnitin, while the majority (77%) were neutral, disagreed or strongly disagreed with the statement ‘I feel anxious about my work being put through Turnitin’ (in the Business group, 73% were neutral or not anxious). This shows that the information sessions were effective at explaining Turnitin to students and reducing their anxiety about putting their work through the software. While 20% of students felt that the practice session ‘helped with their writing’, 31% disagreed (43% neutral). This may indicate that students did not link the wording of the question i.e. ‘writing’ with the academic skills they acquired, as comments received from 54% of the group indicated that: • The originality report was useful; I was glad to use it; • This helped with quotes and referencing (several responses); • It’s a good way to encourage students to use their own words; • It helped significantly with proper referencing; • It showed me how to reference without plagiarism; • The level of plagiarism I had ; • It was useful to see how my work matched up with other texts. Comments made during the video interviews further emphasised how the text-matching colours helped students to realise whether their paraphrasing was accurate or not, whether their work was ‘fine’ and that they may have plagiarised unknowingly. In this way, the colour coding is useful to highlight text but, as pointed out by Davis (2007), using the actual percentage or the colour band is not beneficial. Turnitin groups the percentage index of the Journal of Learning Development in Higher Education, Issue 2: February 2010 7 Cohen Using Turnitin as a formative writing tool originality reports into five bands, for example, blue or green for low percentages. This does not actually reflect whether plagiarism has been found or not. Interestingly, survey results indicated that students felt that Turnitin should be used to discourage plagiarism (60%) rather than as a way to teach students about referencing (just over 20%), an attitude expressed in the follow up interviews which indicated that students felt that Turnitin and reducing plagiarism enhanced the value of their degree, and that Turnitin was a useful deterrent for students. This attitude was expressed in response to the interview question ‘What would you recommend to staff?’ where students indicated that it should be used on every student, in every module and that it should be a policing tool with only one copy to be handed in. A minority felt that Turnitin should be available to students; however, anecdotal evidence from students showed that unlimited access to the originality report encouraged students to try to ‘beat the system’. In terms of advice for other students, responses encouraged others to ‘take the time to use it’ as Turnitin is ‘a helpful way to avoid plagiarism’. Interestingly, this question also raised the point that Turnitin ‘is necessary to ensure University standards’ and that although it ‘seemed daunting at first… it’s beneficial’. On the whole, students appreciated the information sessions on Turnitin and the originality report and the information both reduced their anxiety about using the software and improved their understanding of referencing and acceptable paraphrasing. Staff in both courses reported reductions in cases of plagiarism and one staff member also reported significant reductions in the time taken to investigate possible matching text. See Appendix 1 and Appendix 2 for further detail of the student survey results. Implications of research The sample sizes used in this study were not large as this was an evaluation of a pilot of software. Results indicate that students benefit in several ways from the information sessions on Turnitin, and that supervised time spent examining the originality reports is necessary to facilitate a student’s understanding of what it means to use academic sources in their own words, rather than relying on poor paraphrasing or direct quotations. It is also evident from Journal of Learning Development in Higher Education, Issue 2: February 2010 8 Cohen Using Turnitin as a formative writing tool interview and survey comments that in addition to structured support in using the originality report, students should have the opportunity to check their draft essays on Turnitin, but not have unlimited access to the software, to prevent the possibility of ‘fine-tuning’ their work to avoid plagiarism detection without due acknowledgement of sources, a concern voiced by both staff and students during interviews. Using Turnitin formatively necessitates a certain amount of class time devoted to discussing the originality reports rather than allowing students free (or even limited) access to the software without structured support. Future investigations into how students use the report and whether their writing and referencing skills improve over time and transfer to different types of writing in different modules would be useful for staff who wish to use Turnitin as an academic writing tool. Students have shown that they are happy to work with Turnitin and appreciate its use given sufficient information and support in using the software. Therefore, it is felt that closer adherence to the University guidelines on using Turnitin, in particular the aspects dealing with providing formative experiences of the system, would be beneficial. References Carroll, J. (2002) A handbook for deterring plagiarism in higher education, Oxford: Oxford Centre for Staff and Learning Development. Chester, G. (2001) Plagiarism detection and prevention: final report on the JISC electronic plagiarism detection project. ONLINE: http://www.jisc.ac.uk/uploaded_documents/plagiarism_final.pdf (Accessed: 5 October 2006]. Davis, M. (2007) ‘The role of Turnitin within the formative process of academic writing: a tool for learning and unlearning?’, Brookes eJournal of Learning and Teaching 2(2). ONLINE: http://bejlt.brookes.ac.uk/article/the_role_of_turnitin_within_the_formative_process_of_ academic_writing (Accessed: 30 October 2007). Journal of Learning Development in Higher Education, Issue 2: February 2010 9 http://www.jisc.ac.uk/uploaded_documents/plagiarism_final.pdf http://bejlt.brookes.ac.uk/article/the_role_of_turnitin_within_the_formative_process_of_academic_writing http://bejlt.brookes.ac.uk/article/the_role_of_turnitin_within_the_formative_process_of_academic_writing Cohen Using Turnitin as a formative writing tool Johnston, B. (2003) The concept of plagiarism. The Higher Education Academy. ONLINE: http://www.heacademy.ac.uk/resources/detail/id428_concept_of_plagiarism (Accessed: 18 October 2007). McDowell, L. and Brown, S. (2001) Assessing students: cheating and plagiarism. The Higher Education Academy. ONLINE: http://www.heacademy.ac.uk/resources/detail/id430_cheating_and_plagiarism (Accessed: 22 August 2006). Peacock, S. and Sharp, J. (2006) ‘A Holistic approach to deploying TurnitinUK that encourages sustainable change in institutional plagiarism practice’, pp. 237-238, in Duggan, F. (ed.) 2 International Plagiarism Conference 2006 Proceedingsnd : 19-21 June, 2006. Newcastle: Northumbria Learning Press. Sivasubramaniam, S.D. (2006) ‘Assisting students to avoid plagiarism: the role of formative workshops’, pp. 211 – 219, in Duggan, F. (ed.) 2 International Plagiarism Conference 2006 Proceedings nd : 19-21 June, 2006. Newcastle: Northumbria Learning Press. Author details Judy Cohen is a Curriculum and Educational Development Officer at UELT and joined the department in April 2006. She has taught English as a Foreign/Second language at undergraduate and graduate level in Australia, Sweden and Hong Kong. She has also taught business subjects at FE and undergraduate level. Judy's recent academic background is in education, and she completed her M.Ed. (TESOL) in 2001. Since then she has published in the areas of discourse analysis and plagiarism. Journal of Learning Development in Higher Education, Issue 2: February 2010 10 http://www.heacademy.ac.uk/resources/detail/id428_concept_of_plagiarism http://www.heacademy.ac.uk/resources/detail/id430_cheating_and_plagiarism Cohen Using Turnitin as a formative writing tool Appendix 1 Survey of Business students The university is using the Turnitin Plagiarism Detection Software for the first time this term. As part of the evaluation of Turnitin, students in courses which are using Turnitin are being asked to complete the following questionnaire. The information you provide is important. Please answer the following questions by placing a on the scale below. 1. I have received enough information about Turnitin. strongly agree 22% agree 63% neutral 10% disagree 5% strongly disagree 0 2. I am confident that I understand why Turnitin is being used. strongly agree 41% agree 49% neutral 7% disagree 2% strongly disagree 0 3. I am anxious about my work being put through the Turnitin system. strongly agree 0 agree 27% neutral 29% disagree 32% strongly disagree 12% 4. I know how to get more information about Turnitin. strongly agree 2% agree 61% neutral 22% disagree 10% strongly disagree 5% Journal of Learning Development in Higher Education, Issue 2: February 2010 11 Cohen Using Turnitin as a formative writing tool 5. Comments: Summary of comments received Turnitin: - should be available to students. - should be used in every subject. - should only be used if it adds value to our academic standards. - is necessary to ensure university standards (many similar comments). - seemed daunting at first but now it has been explained I think it’s a good idea and beneficial to students. - will help employers and the university take students more seriously. - good idea to stop lazy students from cheating. - good idea and helpful way to avoid plagiarism. - adds value to students and lecturers. Appendix 2 Survey of Economics students The university is using the Turnitin Plagiarism Detection Software for the first time this term. As part of the evaluation of Turnitin, students in courses which are using Turnitin are being asked to complete the following questionnaire. The information you provide is important. Please answer the following questions by placing a on the scale below. 1. I have received enough information about Turnitin. strongly agree 11% agree 83% neutral 3% disagree 3% strongly disagree Journal of Learning Development in Higher Education, Issue 2: February 2010 12 Cohen Using Turnitin as a formative writing tool 2. I found it useful to have practice sessions in using Turnitin. strongly agree 17% agree 54% neutral 26% disagree 3% strongly disagree 3. The practice session with Turnitin helped me to improve my writing. strongly agree agree 20% neutral 43% disagree 31% strongly disagree 6% By looking at my originality report, I learned that (summary of responses): • I hadn’t plagiarised (7 responses). • Surprisingly it was fine and no useful info coming back. • How many references I used. • There will be a certain amount of matching text. • Turnitin is useful to see how my work matches other texts. • How I should be referencing / time management through earlier submission. • How to reference and quote properly (7 responses). • I could have plagiarised unknowingly. 4. I am anxious about my work being put through the Turnitin system. strongly agree agree 23% neutral 51% disagree 23% strongly disagree 3% Journal of Learning Development in Higher Education, Issue 2: February 2010 13 Cohen Using Turnitin as a formative writing tool 5. Turnitin is best used as (please circle the most appropriate answer): 1) A way to discourage students from plagiarising 60% 2) A way to teach students about referencing 23% 3) A tool that can help students improve their academic writing 11% 4) Other (responses) 6% • A tool for students to check their own references • Reduces the deadline for submission Comments (summary of comments received): - The originality report was very useful. I’m glad I was able to see it. - Be specific about processing time for essays. - Helped significantly with proper referencing of quotes (several comments). - It prevents students using sources without referencing. - It’s ok but didn’t pick up references or quotes in my essay. - Didn’t make much difference and annoying that you have to finish the essay before the deadline (several similar comments). - It’s sometimes accurate but people tell me it misses some quotes. It’s a weak plagiarism deterrent. - Some of the references it finds aren’t the ones I used (several responses). - It improved my writing style and layout of my essay. - It does improve academic writing in trying to integrate referencing in order to make essay more plausible to the argument. - It’s a very useful tool and should be used often (several comments). - Good way to encourage students to use their own words. - It should be available to students. - It should be a policing tool only. - It clarifies your mistakes. Journal of Learning Development in Higher Education, Issue 2: February 2010 14 Using Turnitin as a formative writing tool Abstract Context Research questions Brief methodology Discussion of findings Staff Students Student survey results Implications of research References Author details Appendix 1 Survey of Business students Appendix 2 Survey of Economics students