Literature Review Journal of Learning Development in Higher Education ISSN: 1759-667X Issue 7: June 2014 Improving the quality of academic reflective writing in nursing: a comparison of three different interventions Marion Bowman University of Leeds, UK Berni Addyman University of Bradford, UK Abstract Students are rarely explicitly taught how to develop their writing within a subject discipline, as there is usually a focus on teaching content. However, academic writing, and in particular Academic Reflective Writing (ARW), is very challenging for most students. In this study, a series of three embedded writing development interventions were trialled with successive cohorts of postgraduate nursing students writing a summative 4000 word piece of ARW. The interventions included the use of example texts to make task requirements more explicit, formative peer feedback on draft texts and facilitating increased dialogue between staff and students regarding expectations of this task. Overall, the interventions represented a shift towards assessment for learning. Quantitative results showed a decrease in the number of students investigated for plagiarism, a rise in pass rates and mean grades, and an increased uptake of academic supervision over the three cohorts. In addition, complementary findings from a self-selected focus group interview indicated that respondents perceived the writing development activities to be very useful. In particular, the formative peer and tutor review of written drafts, was valued. However, a limitation of this pragmatic mixed method study was that the three cohorts were non-equivalent. Despite this, it is argued that, as ARW is so complex, disciplinary academics should embed explicit guidance and scaffolding in their teaching in order to enhance written reflection and learning. Failure to do so may lead ARW to become an exclusive educational practice leading to unintentional plagiarism and poor written reflection on practice. Keywords: Academic Reflective Writing (ARW); nursing and midwifery; plagiarism; formative assessment; peer review of written work. Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 2 Introduction In common with academic writing in general, the key features of Academic Reflective Writing (ARW) are usually not explicitly taught by disciplinary academics (Rai, 2006; Kember et al., 2008). This may be due to the fact that there is often an assumption by staff across disciplines that students are already accustomed to academic writing on entry or will pick up the skill required relatively quickly (Gimenez, 2008; Skinner and Mort, 2009; Dyment and O’Connell, 2011). Unfortunately, this runs counter to research on students’ experiences which shows that academic writing is amongst the most challenging and confusing tasks they face at university (Gimenez, 2008; Skinner and Mort, 2009). Research shows that the ability to express ideas in writing develops iteratively over time through scaffolded practice accompanied by detailed, constructive feedback (English et al., 1999). Giving students explicit guidance and support for writing development within their courses is especially important when students are doing ARW for the first time. ARW, is a type of assessment which is associated with students’ placement experiences in Nursing, Health, Education and Social Work (Gimenez, 2008; Ryan, 2011). The aim of ARW is to assess students’ ability to reflect on their strengths, weaknesses and areas for improvement from their experiences on placement, in the light of published literature, professional standards, and a mentor’s observation reports. Thus ARW is neither pure personal diary/journal writing, nor traditional formal academic writing, but has elements of both types, and is one of the most challenging genres of student writing (Gimenez, 2008; Ryan, 2011; Ryan, 2013). Students completing ARW assignments must thus move between textual types, from informal personal reflection of an aspect of experience using ‘I’ or ‘my’, to formal descriptions of the chosen phenomenon from the literature, to explanations of the reasons for the practice-theory divide, and a discussion of implications for future practice (Ryan, 2011). Due to its hybrid nature, ARW is a particularly difficult genre of writing to structure, (Gimenez, 2008) and to compound this, ARW is often summatively assessed and is therefore high stakes (Ryan, 2011; Ryan, 2013). Unfortunately, there is currently very little literature on this genre, with Ryan (2011) and Ryan (2013) being key papers specifically focussing on ARW. There is also very little practical guidance available for students attempting ARW (Rai, 2006). As a result, students may follow the guidance set out in the practical literature on how to write traditional academic essays, and this could result in unsuccessful ARW, as the genres are significantly different (Rai, 2006). Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 3 The SLIP course Support for Learning in Practice (SLIP) is a one-semester 30-credit course which aims to equip Nurses and Allied Health Professionals to lead and mentor other professionals in the workplace. Completing the SLIP course is often a pre-condition for promotion. At the time of this study, the SLIP course was assessed by means of a summative 4000 word piece of ARW which accounted for 100% of the module marks (see Table 1). Table 1. The assignment brief for the summative piece of ARW in SLIP. You are required to write a 4000 word assignment in which you must: Write an account of the quality of the learning and teaching experience. This must incorporate your experience of teaching students in a clinical area in addition to consideration of how mentorship can influence the quality of the learning environment. You must also consider your experiences related to the assessment event undertaken in the clinical environment. This should also include consideration of how mentorship and supervision influence the assessment process. You are required to undertake a minimum of 1 formal and 1 informal teaching session. These MUST be observed by your mentor (in practice) or lecturer on the course (in the case of Midwifery) that must provide comments and feedback on the appropriate documentation. These must be handed in as well as the written assignment in which you should reflect (critical reflection for level 6) on:  The strengths and areas for development within each session  The theoretical underpinnings for method, approach and delivery of each session  The extent to which you were able to (critical at level 6) meet students’ needs  The environment and teaching strategy used and appropriateness  Ethical issues around teaching students/patient involvement consent Teaching materials and teaching plans can be used to augment your work and included as an Appendix. Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 4 SLIP students are typically a mixed intake with many of the students having been out of education for a long time. Prior to the start of the collaborative intervention that will be described in this study, students doing the SLIP module had limited access to supervisors for the review of their written work (supervisors generally only commented once on 500 word extracts), and little explicit guidance was given in the course relating to the textual features of the writing. The academic staff member leading the SLIP module at the time approached a learner developer from the central Learner Development Unit as she had a number of concerns. These concerns included: the quality of students’ ARW, the number of investigations for plagiarism, the pass rates and the initial non-submission rate for assignments. A collaborative partnership then developed to address these concerns and this led to the interventions discussed below. The objectives of the collaborative intervention The objectives of the resulting collaborative intervention were as follows: 1. To embed an assessment for learning approach in the module and to explore students’ perceptions of this initiative. 2. To reduce the number of students who needed to be investigated for possible plagiarism. 3. To reduce the non-submission rate for assignments. 4. To improve the pass rate for first attempts at the assignment. 5. To improve the quality of students’ ARW (as assessed by improvements in the mean grades for ARW assignments). The collaborative intervention There were a number of influences that shaped the design of this collaboration. The first was an awareness that integrated discipline-specific writing development is more effective than designing an additional generic writing skills programme outside a course (Wingate, 2006). The second set of guiding principles was the desire to use an assessment for Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 5 learning approach by including more opportunities for formative assessment to provide feedforward towards the completion of the summative task (Sambell, 2011). In addition, the importance of viewing feedback as a situated social practice, rather than a product, was taken as a starting point for the search for ways to increase the dialogue between students, and between students and staff, about writing (Price et al., 2011). Through dialogue, it is extremely important to explicitly foreground what is expected of students when they are producing assessed writing. This includes being explicit about the kinds of outcomes that need to be met, the length and type of assignment, the expected style and language required, and the required level of reflection (Ryan, 2011; Ryan, 2013). Being explicit and articulating expectations about written assignments is key to improving student performance (Nicol, 2010), particularly for non-traditional students, who may be very unclear about what is expected (Lea and Street, 1998). Finally, Moon’s (2001) framework of pedagogical practices for guiding reflection, e.g. using example texts, maximising feedback and dialogue, discussing the use of ‘I’ in reflective writing, were also used in the design of the intervention with three consecutive cohorts. For the purposes of simplicity, the three occurrences of the collaborative intervention will be represented as Cohort 1 (C1), Cohort 2 (C2) and Cohort (C3). The three cohorts prior to the collaborative intervention will be referred to as C-1, C-2 and C-3 respectively, with C-3 being the earliest data set. Three slightly different kinds of intervention were trialled with the three cohorts (C1, 2 and 3). The intervention with C1 was in the form of a one hour interactive session, during which students analysed good and poor student essay examples. The rationale for this was that students often do not share their lecturer’s understanding of an assignment brief or marking criteria (Nicol, 2010). Thus, it is important that requirements are made explicit when the assignment is first set, and using example student essays is a very efficient way to clarify expectations (Sadler, 1989). As this was the start of the collaboration, the session was led by the learner developer and the example essays were from a different subject area, as there had not yet been time to gain access to SLIP example ARW assignments from consenting students. The nursing lecturer was present at this session and answered students’ questions relating to their own assignment. In addition, students were given general guidance regarding time management and using sources when writing assignments. Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 6 The interactive writing session with C2 was more closely aligned to needs of SLIP students. Permission had been gained from students from the previous cohort to use their ARW assignments for teaching purposes. The learner developer then ‘engineered’ these examples to focus students’ attention on the common errors seen throughout the writing of the previous cohort. ‘Engineering in’ common errors into adapted student exemplars is a form of pre-emptive formative feedback (Carless 2007, cited Sambell, 2011). The types of errors included in the reconstructed weak examples of ARW written for student analysis, included: the incorrect use of ‘I’, mismatches between reflection and the literature, inappropriate source use, and poor text structure. Understanding the exact requirements of a written assessment task is an ongoing process as a student’s understanding of the task becomes more refined as work on the assignment proceeds (Nicol, 2010). Cohort 2 were therefore offered a second writing development session halfway through the SLIP module, during which students brought in drafts for formative peer feedback and tutor comment. The peer review session started with a discussion of how to give feedback in a supportive way and then students commented on each other’s drafts, with some making use of the generic peer review checklist in Table 2. Table 2. The peer marking checklist used by pairs of students from Cohort 2 in the second writing development session. Marking Criteria Marks Introduction: general to specific?  follows general to specific pattern  starts with a general statement  states topic / argument in essay clearly  defines topic (optional – could be in first paragraph)  ensures confidentiality of those mentioned  {in long essay: states structure of essay – order of ideas} / 5 Citations (references in brackets in the text)  after every statement that doesn’t come from own head  at least one source has been cited  citations are in the correct format (Surname, date). Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 7  the idea from the literature matches the topic of the idea from personal reflection.  all citations are from quality sources / 5 Paragraphs  all paragraphs are related to the essay title, no irrelevant information  all paragraphs start with a clear topic sentence  all stay on topic – one theme per paragraph  there is a good flow of ideas from paragraph to paragraph  the conclusion is clearly linked to rest of text and to essay title / 5 Other  the conclusion follows the specific to general pattern  there is a strong argument in this text, points are well supported – strong believable conclusion  there is a list of references at end of essay  the list of references is arranged alphabetically  the end reference list is in the correct format / 5 The intervention with C3 was similar to the intervention with C2, in that it consisted of one early interactive session using re-engineered SLIP example texts, followed by a peer review session half way through the module. However, with C3 the second writing development session was more closely linked to the expectations and requirements of the SLIP assignment (compare the peer review checklist in Table 2 to the one in Table 3). In addition, more discussion on how to structure ARW was elicited from students in the second intervention session with C3, as structuring the written task had emerged as a key challenge in C2. Thus the interventions became both more embedded in the course and more explicit through time, from C1 to C3. Another key factor was that the disciplinary lecturer played a progressively greater role in the sessions (answering student questions, giving explicit guidance on the task and taking feedback from text-based activities), as the collaboration progressed from C1 to C2 to C3. Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 8 Table 3. The peer review checklist used by pairs of students from Cohort 3 in the second writing development session. Marking Criteria Check Introduction: general to specific?:  follows general to specific pattern; starts with a general statement, ends with the structure of this essay.  states topic / argument clearly  defines topic (optional – could be in first paragraph)  ensures confidentiality of those mentioned  states the structure of the essay clearly (order of ideas to follow) / 5 Citations (references in brackets in the text):  after every statement that doesn’t come from own head  citations are in the correct format (Surname, year).  all citations are from quality sources  a range of evidence has been used, including: personal evidence, evidence from the mentor, evidence from the literature, evidence from regulatory bodies. / 4 Paragraphs:  Each paragraph stays on one topic / aspect and in each paragraph, all the evidence (personal experience, the literature, mentors comments, the NMC code / regulations) matches the one aspect being discussed in the paragraph.  all paragraphs start with a clear topic sentence. Paragraphs are analytical and evaluative; interpreting the teaching / assessment event, rather than just describing it. / 3 Marking Criteria Marks Content:  Both the strengths and weaknesses of the teaching or assessment event have been discussed (with evidence).  The observations notes and other documents relating to the teaching and assessment events will be included in an Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 9 appendix and are referred to in the paragraphs e.g. (see Appendix 1).  The extent to which your partner was able to meet their students’ needs has been evaluated.  Your partner has discussed the theoretical justification for the teaching / assessment approach they have chosen.  Your partner has considered the appropriateness of the teaching environment and strategy.  Your partner has considered the ethical issues around teaching students / patient involvement and consent. / 6 At the end of the essay:  There is a strong argument in this text. The points are well supported and lead to a strong and believable conclusion.  There is a list of references at end of essay and it is arranged alphabetically and is in the correct format. / 2 Method A pragmatic mixed methods approach (Teddlie and Tashakkori, 2009) was used to evaluate whether the five objectives of the collaborative intervention were met or not. Descriptive quantitative data on contrasting pass rates, non- submission rates, the number of student visits to supervisors and overall grades, for C1, C2 and C3 (as well as for C-1, C-2 and C-3), were compared. The distributions of grades for C1, C2 and C3 were also tested for normality using the Kolmogorov and Smirnov and Shapiro-Wilk tests, which are powerful tests of normality (Razali and Wah, 2011). The means of all six cohorts were compared using a two-tailed independent samples t-test to test for any significant differences between means. As can be seen in Table 4, the different iterations of the course occurred at contrasting times of the year with differing numbers of students in each cohort. Thus, these comparisons represent a post-test only non-experimental design (Gravetter and Forzano, 2009). However, the academic regulations, entry requirements, academic supervisors and markers for C1, 2 and 3 remained the same. Ethical approval was granted prior to the commencement of this study. Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 10 Table 4. Characteristics of the six cohorts (with C1, 2 and 3 being during the period of the collaborative intervention and C-1 -2 and -3 being prior to the intervention). The semester in which the course was run Number of students in Cohort Cohort 1 Semester 1 89 Cohort 2 Semester 2 45 Cohort 3 Semester 1 43 Cohort -1 Semester 1 108 Cohort -2 Semester 2 118 Cohort -3 Semester 1 39 The quantitative data comparisons discussed above were complemented by qualitative data from a focus group interview with a consenting self-selected group of eight students from C3. The eight participants consisted of six females and two males from the full range of ages within the cohort, as well as three students who had English as a second language, and at least two who had been out of education for a long time, out of a cohort of 39. This group of students was representative of the diversity of students in the cohort, however, it is possible there was a slight bias towards students who had greater difficulty doing ARW. The interview was conducted by an independent researcher in a nearby teaching room just after the students had finished the peer review intervention session. The researcher started the interview with an open question about how students were finding the course, and then moved the discussion along to explore students’ perspectives of the assessment for learning approach following in this module. It was thought that this qualitative data might bring a richer understanding of students’ opinions and responses to the interventions (Kitzinger, 1995; Kitzinger and Barbour, 1999). The focus group interview was transcribed verbatim by the first author of this paper. During transcription, features such as overlapping speech, inaudible sections, stuttering, murmured agreement, pauses, interruptions and laughter, were noted, as any of these features could carry further meaning as they could be linked to consensus or disagreement (Kitzinger and Barbour, 1999). The final transcript was then sent to the independent researcher who originally conducted the focus group interview for verification as an accurate record of what transpired. Later the raw quotations were tabulated in Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 11 emergent categories and then a concept map showing the links between categories and the strength of different responses was drawn up (see Figure 1). The original quotations that were thought to best represent particular categories of response have been used in the discussion section of this paper. Figure 1. A concept map showing the major themes emerging from the focus group interview with C3. Useful: reassurance (2) Useful: clear explanations / pitched at right level (2) Online links to key readings on VLE (1) Useful: peer review of drafts in session (2) Useful: referencing (1) Useful: structuring writing (4) Scaffolding and support: usefulness The writing development sessions Very useful generally (3) Useful: consolidation / way forward (1) Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 12 Results and discussion Aim 1: To embed an assessment for learning approach in the module and to explore students’ perceptions of this As has been discussed above, ARW is a particularly challenging genre of writing. This challenge is further heightened if students are unfamiliar with academic writing, e.g. postgraduate students or first year students, and if they are operating in a context that involves significant time pressure, e.g. writing ARW assignments whilst on work placement. Thus, an important aim of the collaborative intervention was to embed an assessment for learning approach in the module by using the writing development sessions to: clarify task expectations using student example assignments, provide opportunities for formative peer review, and to increase dialogue between students and the tutor regarding the task. In the focus group interview, there was consensus amongst participants that the writing development sessions had been really useful. A discussion about the usefulness of these sessions emerged spontaneously in the interview, not in direct answer to a question from the interviewer. Participants expressed their perceptions of usefulness in a number of ways (see Figure 1), starting with a general discussion: …sessions have been, really, really good (murmured agreement), really, really useful… (sound of hand hitting table) bang on the mark, hitting the nail on the head, almost like ‘ah’, almost like a little eureka moment. (Female 1) (Where ( ) = audible sounds) When asked to explain what, in particular, participants found useful about the sessions, the discussion turned first to how the sessions helped respondents to structure their ARW, with four of the eight respondents making this link: For me, [what was useful about the sessions] it’s definitely structuring my essay (mmm murmured agreement) and flow. She [the tutor] said well, ‘how about you think about talking about one element and then drawing in bits?’ and we were like ‘oh, no, no, I don’t like that’, and then when she explained it we were like ‘oh, that makes more sense’, like, ‘it makes much more sense’, so she’s opened your eyes to different ways… [of structuring the piece]. (Female 1) (Where [ ] = text added for clarity) Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 13 And Where I’ve gone off on a tangent, she’s [tutor] gone – ‘what’s this bit here…?’ (Female 2) During the plenary section of the second intervention session for C3, students had asked a lot of questions about how to link their ideas and structure their ARW piece, and a lively discussion had ensued. The lecturer and learning developer had also responded to students’ individual queries about ordering their ideas whilst monitoring the peer review activity. There is research to show that such dialogue between tutors and students has the potential to lead to significant improvements in the quality of resulting written assignments (Nicol, 2010; Ryan, 2011), with the corollary that better quality writing represents a greater clarity of thinking. The focus group respondents also linked the perceived usefulness of the intervention to the clarity of explanations and the level at which the writing development sessions were pitched. …there are a lot of us that haven’t [been in academia recently], and it’s pitching it at that level that everybody understands, and I really did think that today was really, really good. (Female 4) The quotation above illustrates the respondent’s desire for explicitness and clarity. It is crucial that disciplinary staff are explicit about what is required of students for each assessed task (Nicol, 2010). This is particularly the case for ARW as students are often very unfamiliar with systematic written reflection (Rai, 2006; Hamilton and Druva, 2010). One focus group respondent linked the usefulness of the focus group interview to guidance on how to reference and use sources, and two students mentioned the peer review of draft reflective essays as a useful feature. I haven’t massively brought something really good, but it’s… encouraging that you’re on the same path as your friends (agreement: yeah yeah). (Female 1) Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 14 The quote above also seems to indicate that this respondent found the session reassuring. This was echoed by two other respondents, one of whom expressed that the session had consolidated what she knew and set out the way forward. It may be the case that even students who have had recent academic experience could gain some reassurance from having someone read and comment on their draft assignment. I thought it [the draft] were horrendous really, but now I just feel a bit more positive today, after having this session. (Female 2) Aim 2: To reduce the number of students who had to be investigated for possible plagiarism One of the aims of the collaborative intervention discussed in this study was to reduce the number of students that had to be investigated for plagiarism, through integrating opportunities for explicitness, dialogue and formative feedback into the module. The assumption underlying this aim was that some or all of the plagiarism was unintentional, as the concept of plagiarism is often a novel one, especially for novice academic writers and non-traditional students (Ashworth et al., 1997; Pecorari, 2003). In learning how to use sources effectively in writing, students may not be able to visualise how to avoid plagiarising (Ashworth et al., 1997) and may either lean too heavily on sources, not attribute ideas to authors accurately or engage in incomplete paraphrasing (Pecorari, 2003). The results in Table 5 show that plagiarism rates did reduce during the period of the collaborative intervention (C1, C2 and C3 as compared to C-1, C-2 and C-3 prior to the intervention). The following comment from a student who had been out of education for some time, illustrates the point that non-traditional students may face significant challenges when it comes to attributing sources: …and with me [what’s been useful] it’s the referencing, ‘cos I’m not good, you know, I know they’ve to be there but I think I’d put those brackets anywhere I felt like it (laughter) or didn’t as the case may be, you know, it were a just a jumble…And now [after the second writing session] I can order the references a bit more. Hopefully it will be much better. (Female 2) Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 15 Table 5. The number of students given a mark of zero and investigated for possible plagiarism, both prior to the collaborative intervention (C-1, C-2 C-3) and during the intervention (C1, C2, C3). Cohort Number of students investigated % of investigations per cohort Explanation/details C-1 3 investigations 3% Collaborative intervention not yet begun C-2 5 investigations 4% Collaborative intervention not yet begun C-3 1 investigation 3% Collaborative intervention not yet begun C 1 2 investigations 2% Both students attended the one initial writing development session. C 2 0 investigations 0% The collaborative intervention consisted of the initial session and the second session in which drafts were peer marked. C 3 1 investigation 2% This student was absent for both the initial development session and the peer marking session. It is interesting to note that the number of plagiarism cases for cohorts C2 and C3 was lower than the rate for C1. It is possible that existence of the second writing development session during which students were able to receive informal advice on their drafts from peers and tutors, may have made a difference to the plagiarism rates. There is support for this in the literature. According to Wheeler (2009) and Pecorari (2003) it is extremely important for novice academic writers to be given the opportunity to practise using sources in writing and to receive detailed and constructive feedback on their attempts. Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 16 In addition, two focus group respondents, in discussing which aspect of the writing development intervention they found useful (see Figure 1), specifically singled out the peer review session as being useful for their writing development: …the split sessions as well, I honestly didn’t think I’d bring anything to session and… I think if we had just one [session] at the beginning… we could have quite easily forgotten (agreement: yeah yeah). (Female1) It has been really helpful when we had the draft copies, last time we came and we read other people’s work. (Female 5) Aim 3: To reduce the initial non-submission rate for assignments It was hoped that the increased explicitness and dialogue put in place as part of these interventions would help to reduce initial non-submission rates for this assignment. (A number of students had not been handing in their assignments on time, but instead chose to absorb a financial penalty and hand in a bit later instead; their assignments were still awarded a mark). Figure 2. The initial non-submission rates for the assignment (C1-3). Figure 2 above shows an interesting pattern that may be linked to the ten week nature of the SLIP course. Cohort 2 (who attended the course in semester 2 rather than semester 1) showed the lowest non-submission rate. A similar pattern can be seen in the three cohorts Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 17 prior to the intervention discussed in this study, with C-2 (a semester 2 cohort) having a much lower initial non-submission rate (C-2=6% as compared to C-1=18%, and C-3 = 12%). In addition, whilst C-1, C-2 and C-3 did show higher non-submission rates than C1, C2 and C3, this pattern varied according to when the course was run. The semester 1 or Christmas term occurrences of the course generally had higher initial non-submission rates. Aim 4: to improve the pass rate for first attempts at the assignment A further aim of the intervention in this study was to improve the pass rate for first attempts at doing this assignment. It was thought that the increased explicitness about the task, the facilitation of dialogue about the task once the students had produced a draft, and the peer review opportunity, might lead to greater number of students passing first time around. Figure 3 shows that pass rates did improve progressively over the three cohorts, however as these are non-equivalent cohorts, it is not possible to definitively say that this was the direct result of the writing intervention. Figure 3. The % of students passing their assignments on the first attempt. In addition, prior to this intervention, the first submission pass rates had been 49% Cohort -3, 52% for Cohort -2 and 55% for Cohort -1. Thus there seems to be a general trend towards improvement over time. It must be borne in mind that each cohort was distinct and comprised different numbers of students (see Table 3). It is also interesting to note that the number of students taking the opportunity to visit their supervisor also showed an Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 18 increasing trend over time (see Figure 4) and it is possible that these supervisor visits may also have been a factor contributing to the increased pass rates. (At the time of this study, students were allowed to visit their supervisor once with a 500 word section of their draft. Not that many students had been taking the opportunity to make an appointment with their supervisors). It is possible, though, that the writing development sessions played a role in giving the students the confidence and encouragement to seek supervision. The following quotation illustrates the uncertainty that two of the focus group participants felt towards approaching their supervisor. I purposely haven’t booked my appointment with my academic supervisor yet, ‘cos I don’t feel that I’m at a stage yet where I can gain enough benefit from their input (murmured agreement: female student), plus the fact that they are only allowed to look at five hundred words… well somebody’s already done that for me today [second writing development session] …and I‘ve gained some changes that need to be done about style, format and everything else, so the fact is that I’m quite pleased that I’ve not wasted me session with me academic supervisor… yet. (Female 6) The quotation above also seems to illustrate that the student felt more confident about visiting their supervisor and that the increased dialogue with fellow students and a tutor in the writing session had been useful. Figure 4. The % of students visiting their supervisors (C1-C3). Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 19 In contrast, only a relatively small proportion of students took the opportunity to visit the university’s central Learner Development Unit (LDU) with drafts of their written work (this offer of additional advice was promoted during the intervention sessions). Only 6% of C1 visited the LDU for advice, in contrast to C2, where 9% of students accessed this central unit. A very small percentage (2%) of students in C3 visited the LDU. Given the small number of students taking up the offer of additional advice on their writing, it is hard to determine the degree to which these visits to the centralised unit may have influenced pass rates. Aim 5: To improve the quality of students’ ARW (as assessed by improvements in the mean grades for ARW assignments) Figure 5 shows the spread of grades for students in Cohorts 1, 2 and 3. As illustrated in this graph, the mean grades (shown by the bar in the centre of the box) increased with each successive intervention. In addition, the median scores also surpassed the mean grade in C2 and C3 (see Table 6), which indicates the general trend towards higher grades. In addition, the spread of results decreased from C1, to C3, which may be an indicator of a greater consensus of understanding on the expectations of the assessed task. The outlying points linked to C1 and C3 show the number codes for individual students who either were investigated for plagiarism or got very low results for their assignments. The spread of grades for Cohorts 1, 2 and 3 represented in Figure 5 were tested for normality using the Kolmogorov-Smirnov and Shapiro Wilk tests. Cohorts 1 and 2 were normally distributed, however Cohort 3 was not normally distributed at the 0.05 significance level. Cohort 3 does represent a smaller student group than the previous two Cohorts. Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 20 Figure 5. Box plots showing the spread of results for C1, C2 and C3. The horizontal line in the centre of the box indicates the mean grades and the vertical lines showing the extent of the spread of grades. Table 6. The mean and median grades for the three cohorts prior to the intervention (C-3, C-2, C-1) and during the intervention (C1, C2, C3). It was found that Cohorts 1 and 2 were not significantly different from each other, with a p value of 0.831 for a two-tailed independent samples t-test where equal variances are assumed (as illustrated in the box plots in Figure 8). However, C2 was found to be significantly different from C3 (p= 0.22) and C1 was also found to be also significantly Cohort Mean Median Cohort 3 51% 59% Cohort 2 51% 54% Cohort 1 48% 48% Cohort -1 39% 43% Cohort -2 41% 43% Cohort -3 49% 45% Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 21 different from C3 (0.08). In terms of the three Cohorts prior to the intervention; C -1, C-2 and C-3 were not significantly different to each other at the p=0.05 significance level. There was also no significant difference between the cohort immediately prior to the intervention (C-1) and the first cohort of the intervention: C1 (p=0.401). This data seems to indicate that there was something a bit different about the spread of grades in C3. It is difficult to determine whether this was due to the smaller size of the cohort or the effect of the intervention. Conclusion In conclusion, in this paper, three slightly different embedded writing development interventions were evaluated. The interventions, with three successive cohorts of students doing a post-graduate nursing course, were designed in order to try to improve the quality of the summative ARW assignment associated with this course, and hence the quality of reflection and learning. Results showed that pass rates for this assignment, mean grades and student visits to supervisors followed an increasing trend from the first to the third cohorts, whilst plagiarism rates declined. The non-submission rate for assignments, however seemed to vary according to whether the course was run during the Christmas or summer terms. A limitation of this study is its non-empirical design, with different cohorts of students being compared, as is often the case in educational research. Thus, it is not possible to definitively attribute the quantitative gains in this study to the incremental adoption of an assessment for learning approach, but it is likely that the intervention played a role. The focus group interview showed that the self-selected group of respondents found the intervention sessions to be very useful, in particular for the discussions on how to structure ARW. Respondents also found the peer review and tutor comments on drafts of their work very useful. In addition, the explicit explanations of how to successfully reflect critically in writing in this context, were perceived as helpful. The interventions trialled in this study follow an assessment for learning approach (Sambell, 2011), due to the emphasis on formative, low-stakes and informal feedforward prior to the formal summative assessment. In this study, feedforward was conceptualised as a two way dialogic process (Nicol, 2010) with the writing development sessions Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 22 providing a space for increased dialogue between students, and between staff and students about the assignment. Research shows that the greater the degree of overlap between student and disciplinary staff conceptions of writing tasks, the greater the quality of student writing (Hounsell 1997, cited Nicol, 2010). In fact, many students cannot produce the level of disciplinary writing often expected by academic staff without explicit guidance and formative feedback (Hamilton and Druva, 2010). Not providing guidance and opportunities for development could be thought of as an ‘exclusive educational practice’, especially where non-traditional students are concerned (Rai, 2006), which may result in unintentional plagiarism and poor quality writing and learning, in an atmosphere where students may feel alienated from their tutors (Ashworth et al., 1997). This sense of alienation from tutors while students are working on assessed tasks may be a contributing factor to the consistently low NSS scores that students assigned to feedback (Ashworth et al., 1997). According to Ryan (2013), unless pedagogic scaffolding is provided for students at every level of reflection, (reporting an important aspect of experience [using ‘I’], relating this to the literature or professional standards, exploring reasons for any differences, and reframing future professional practice), gaps in students’ reflection may result. This may mean that students are not able to move to the highest aim of academic reflection on practice, i.e. that their learning and professional practice are reframed and transformed. It is possible that the interventions in our study did not go far enough in providing scaffolding for the highest level of transformative reflection and learning. Many of the students on the SLIP course were at the stage where they were grappling with how to link their experiences to the literature without plagiarising. Providing scaffolding for all the levels of reflection may be challenging to achieve in a content-heavy ten-week course taken by students, many of whom have been out of academia for quite some time. It is proposed that more research on ARW, as a very challenging (Rai, 2006; Ryan, 2011: Ryan, 2013) vehicle for assessing reflection on work placement, is sorely needed. Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 23 References Ashworth, P., Bannister, P. and Thorne, P. (1997) ‘Guilty in whose eyes? University students’ perceptions of cheating and plagiarism in academic work and assessment’, Studies in Higher Education, 22(2), pp. 187-203. Dyment, J.E. and O’Connell, T.S. (2011) ‘Assessing the quality of reflection in student journals: a review of the research’, Teaching in Higher Education, 16(1), pp. 81-97. English, L., Bonanno, H., Ihnatko, T., Webb, C. and Jones, J. (1999) ‘Learning through writing in a first-year accounting course’, Journal of Accounting Education, 17(2), pp. 221-254. Gimenez, J. (2008) ‘Beyond the academic essay: discipline-specific writing in nursing and midwifery’, Journal of English for Academic Purposes, 7(3), pp. 151-164. Gravetter, F.J. and Forzano, L-A.B. (2009) Research methods for the behavioural sciences, 4 th edn. Canada: Wadsworth Cengage Learning. Hamilton, J. and Druva, R. (2010) ‘Fostering appropriate reflective learning in an undergraduate radiography course’, Radiography, 16(4), pp. 339-345. Kember, D., McKay, J., Sinclair, K. and Wong, F.K.Y. (2008) ‘A four-category scheme for coding and assessing the level of reflection in written work’, Assessment & Evaluation in Higher Education. 33(4), pp. 369-379. Kitzinger, J. (1995) ‘Qualitative research: introducing focus groups’, British Medical Journal, 311, pp. 299-302. Kitzinger, J. and Barbour, R.S. (1999) Developing focus group research: politics, theory and practice. London: Sage. Lea M. and Street B. (1998) ‘Student writing in higher education: an academic literacies approach’, Studies in Higher Education, 23(2), pp. 157-172. Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 24 Moon, J. (2001) PDP working paper 4: reflection in higher education learning. Exeter: LTSN Generic Centre, University of Exeter. Available at https://www.york.ac.uk/admin/hr/researcher- development/students/resources/pgwt/reflectivepractice.pdf (Accessed: 23 April 2012). Nicol, D. (2010) ‘From monologue to dialogue: improving written feedback processes in mass higher education’, Assessment and Evaluation in Higher Education, 35(5), pp. 501-517. Pecorari, D. (2003) ‘Good and original: plagiarism and patchwriting in academic second- language writing’, Journal of Second Language Writing, 12(4), pp. 317-345. Price, M., Handley, K. and Millar, J. (2011) ‘Feedback: focussing attention on engagement’, Studies in Higher Education, 36(8), pp. 879-896. Rai, L. (2006) ‘Owning (up to) reflective writing in Social Work Education’, Social Work Education: The International Journal, 25(8), pp. 785-797. Razali, N. and Wah, Y.B. (2011) ‘Power comparisons of Shapiro-Wilk, Kolmogorov- Smirnov, Lilliefors and Anderson-Darling tests’, Journal of Statistical Modeling and Analytics. 2(1), pp. 21-33. Ryan, M. (2011) ‘Improving reflective writing in higher education: a social semiotic perspective’, Teaching in Higher Education, 16(1), pp. 99-111. Ryan, M. (2013) ‘The pedagogical balancing act: teaching reflection in higher education’, Teaching in Higher Education, 18(2), pp. 144-155. Sadler, D.R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional Science, 18(2), pp. 119-144. Sambell, K. (2011) Rethinking feedback in higher education: an assessment for learning perspective. Bristol: ESCalate, Higher Education Academy. Available at: http://dera.ioe.ac.uk/14776/1/8410.pdf (Accessed: 10 February 2014). https://www.york.ac.uk/admin/hr/researcher-development/students/resources/pgwt/reflectivepractice.pdf https://www.york.ac.uk/admin/hr/researcher-development/students/resources/pgwt/reflectivepractice.pdf https://www.york.ac.uk/admin/hr/researcher-development/students/resources/pgwt/reflectivepractice.pdf http://dera.ioe.ac.uk/14776/1/8410.pdf Bowman and Addyman Improving the quality of academic reflective writing in nursing Journal of Learning Development in Higher Education, Issue 7: June 2014 25 Skinner, I. and Mort, P. (2009) ‘Embedding academic literacy support within the Electrical Engineering Curriculum: a case study’, IEEE Transactions on Education, 52(4), pp. 547-554. Teddlie, C. and Tashakkori, A. (2009) Foundations of mixed methods research: integrating quantitative and qualitative approaches in the Social and Behavioural sciences. USA: Sage. Wheeler, G. (2009) ‘Plagiarism in Japanese universities: truly a cultural matter?’, Journal of Second Language Writing, 18(1), pp. 17-29. Wingate, U. (2006) ‘Doing away with study skills’, Teaching in Higher Education, 11(4), pp. 457-469. Author details Marion Bowman is currently working as a Dental Education Adviser in the School of Dentistry at the University of Leeds. Marion used to work in the central Learner Development Unit at the University of Bradford, which is where she met Berni. Berni Addyman is a Mental Health Nursing Lecturer in the School of Health Studies at the University of Bradford. Improving the quality of academic reflective writing in nursing: a comparison of three different interventions Abstract Introduction The SLIP course The objectives of the collaborative intervention The collaborative intervention Method Results and discussion Aim 1: To embed an assessment for learning approach in the module and to explore students’ perceptions of this Aim 2: To reduce the number of students who had to be investigated for possible plagiarism Aim 3: To reduce the initial non-submission rate for assignments Aim 4: to improve the pass rate for first attempts at the assignment Aim 5: To improve the quality of students’ ARW (as assessed by improvements in the mean grades for ARW assignments) Conclusion References Moon, J. (2001) PDP working paper 4: reflection in higher education learning. Exeter: LTSN Generic Centre, University of Exeter. Available at https://www.york.ac.uk/admin/hr/researcher-development/students/resources/pgwt/reflectivepractice.pdf (Access... Teddlie, C. and Tashakkori, A. (2009) Foundations of mixed methods research: integrating quantitative and qualitative approaches in the Social and Behavioural sciences. USA: Sage. Wingate, U. (2006) ‘Doing away with study skills’, Teaching in Higher Education, 11(4), pp. 457-469. Author details