691 Tuzcu-Eken, D. (2021). Peer evaluation in writing: How to implement efficiently. International Online Journal of Education and Teaching (IOJET), 8(2). 691-708. Received : 11.07.2020 Revised version received : 26.11.2020 Accepted : 12.12.2020 PEER EVALUATION IN WRITING: HOW TO IMPLEMENT EFFICIENTLY Research article Derya Tuzcu-Eken Kırklareli University derya.tuzcu@klu.edu.tr Dr. Derya Tuzcu-Eken currently works as an English Instructor in the Department of Foreign Languages at Kırklareli University, Turkey. She holds her PhD in Teaching English as a Foreign Language from Çanakkale Onsekiz Mart University. She is currently interested in the development of intercultural skills in all levels of language learners including young learners, the advancement of writing skills of adult learners, and distance education of English language. Copyright © 2014 by International Online Journal of Education and Teaching (IOJET). ISSN: 2148-225X. Material published and so copyrighted may not be published elsewhere without written permission of IOJET. mailto:derya.tuzcu@klu.edu.tr https://orcid.org/0000-0003-2606-1314 Tuzcu-Eken 692 PEER EVALUATION IN WRITING: HOW TO IMPLEMENT EFFICIENTLY Derya Tuzcu-Eken derya.tuzcu@klu.edu.tr Abstract The aim of this paper is to investigate the most efficient ways to apply peer evaluation in writing courses, and a lack of studies on the views of students’ experienced difficulties while responding to their peers’ papers is the starting point of this paper. For this cause, the case study method was implemented, including an interview conducted with prep-class students (N=37) who attended a writing course for two semesters and a review of literature of related papers written over the past five years that specifically focuses on the ways to develop peer- evaluation skills of students. Both the interview and the relevant previous studies were analyzed through content analysis. As a result of the study, it was found that the challenges that students experienced were because of a lack of ability in language usage and having an avoidant personality, not having clear purposes for evaluation, a lack of ability in dominating the relevant terminology, and having difficulty in finding content and organization related mistakes. Consequently, students need effective training on peer evaluation to enhance their skills beforehand. The suggestions collected thanks to the literature review is expected to be helpful in this respect. Keywords: peer evaluation, peer assessment, peer feedback, writing course 1. Introduction As one of the most attention-grabbing topics of writing instruction, peer evaluation is one type of feedback that has been used in writing courses. Although teacher feedback is still the most prevalent and highly valued feedback type for the students (e.g., Ferris & Hedgcock, 2005; Lee, 2008) peer evaluation has started to gain importance, especially with the shift from teacher-centeredness to the student-centeredness in educational environments. Peer evaluation, in other words, peer assessment, as a form of collective learning, hinges on students’ assessment of each other’s papers. To put it in detail, peer evaluation is “an educational activity in which students judge the performance of their peers, and it can take different forms depending on the characteristics of its implementation, the learners and the learning context” (Alqassab & Panadero, 2020, p. 2). It has various aims ranging from summative to formative objectives (Panadero, Jonsson & Alqassab, 2018). Formative peer assessment, as its very name signifies, relies on ongoing assessment of the students’ papers and requires students to evaluate their peers’ papers before the final product is released. Summative peer assessment, on the other hand, is the evaluation of the final product in which students evaluate each other’s papers after they finish writing it. Both types of assessment can be used separately or together, depending on the purpose of the assessment. They are all beneficial in the development of writing skills of students as well as their self-autonomy because peer evaluation “encourages students’ autonomy and higher-order thinking skills” (Bostock, 2000). However, “in an educational setting, most peer assessment consists of a single round” (Song, Hu, Guo & Gehringer, 2016), and as students themselves are also the learners of the writing skill, it sometimes gets difficult for them to correct each other’s mistakes and comment on their peers’ papers. Therefore, students need to be trained by their course teacher on how to mailto:derya.tuzcu@klu.edu.tr International Online Journal of Education and Teaching (IOJET) 2021, 8(2), 691-708. 693 evaluate a piece of writing by focusing on language (grammar mistakes, vocabulary choice, etc.), content (appropriateness of the text to the topic, and finding out irrelevant sentences), and organization (title, topic sentence, supporting ideas, concluding sentence) of the paper as well as repeating the process with formative and summative peer assessments. Besides, as revealed in the study of Zhao (2018) teachers have limited knowledge of peer evaluation because of their lack of enough training in peer evaluation, so in some situations, teachers also need training on how to implement peer evaluation in writing classes. As a novice teacher of writing, the researcher herself had the same difficulty in applying peer evaluation, and she was concerned about the negative views of her writing class students on peer evaluation exercises conducted in class. After being assigned as their course teacher, she started to search for materials to teach writing appropriate to the level. She valued both teacher and peer feedback, yet during the first term in order to let them get used to the process of writing, she did not give any peer evaluation assignments. In the second term of the course, she believed that they were ready for the process as she expected them to have learnt paper evaluation from her evaluations of the students’ papers. However, in time she understood that most of the students were not aware of how to conduct peer evaluation, and they needed critical guidance on peer evaluation. Then, she prepared handouts for them, which included some instructions for students to follow while they evaluate their peers’ papers. Some of the peer evaluation training activities seemed to be received well, yet sometimes she thought that students got bored and found the peer evaluation exercises unnecessary. In order to have more efficient peer evaluation sessions for the following years, the researcher conducted this study. Therefore, the current study aims to examine the most efficient ways of implementing peer evaluation in a writing course through querying the views of students on the difficulties that they experienced while doing peer evaluation in their writing course. The studies analyzed for this cause are expected to compare the views of participant students with the findings and suggestions of similar studies. In this way, it is expected to compensate for the lack of knowledge of the course teacher and focus on the needs of students in peer evaluation. The research questions of the study are as follows: 1. What are the views of prep-class students on the difficulties that they experienced during peer evaluation practice conducted in a writing course? 2. What does recent research on peer evaluation suggest for a more applicable implementation of peer evaluation in writing classes? 2. Method 2.1. Research Design The current paper employs a case study research design. A case study is a research method that involves an in-depth analysis of a case which is composed of a case or multiple cases. It is, in a way, “the study of a social phenomenon” (Swanborn, 2010, p. 13). As stated by Gerring (2004, p. 342), a case study is “an intensive study of a single unit for the purpose of understanding a larger class of (similar) units.” In a case study, if the aim is to investigate a case (composed of a group of people) asking retrospective questions to understand the things that happened in the past, or the process itself is beneficial in giving insights about the phenomenon (Swanborn, 2010). 2.2. Setting and Participants The participants of the study are 37 prep-class students: 25 females and 12 males. These students are preparatory class students of an English language and literature department at a Turkish university. Their English level ranges between intermediate to upper-intermediate. Tuzcu-Eken 694 However, they could not pass the English proficiency exam of the university, carried out at the beginning of the school year, because of a lack of writing skills. They took the writing course for two semesters, and especially during the second semester, the course teacher assigned them some tasks to evaluate each other’s papers. These peer evaluation assignments were generally summative. However, during these activities, it was observed that the students were not so much aware of the benefits of peer evaluation activities and it was so difficult for them to evaluate another paper even if the course teacher had made explanations on how to evaluate, where to focus on, and what the benefits of these evaluations are. 2.3. Data Collection To collect data, an open-ended interview designed by the researcher was utilized to probe the ideas of the students on the difficulties of peer evaluation, and in this way, it was expected to find an answer to the first research question. The interviews were conducted in Turkish in order to eliminate the stress that may result from not remembering necessary vocabulary or language incompetency. Moreover, in an attempt to find answers to the problems of the participant students and to see what there is in the literature to benefit from, analysis of the previous literature was put to use, which also aimed to answer the second research question. The studies that were selected to be analyzed were the articles related to the aims of the research and were the most recent ones published between the years 2015-2020. However, as it necessitated, a few studies conducted before the year 2015 were included. While choosing the articles, “peer evaluation in writing,” “peer assessment in writing,” “peer feedback in writing,” search terms were used to attain the most relevant papers from the prospective journals. As a result, 39 articles were selected from the 297 articles found. Then, with in-depth analysis, the number of articles was decreased to 22. 2.4. Data Analysis To analyze the data collected, a qualitative content analysis method that provides an in- depth analysis of an issue by deducing the categories and themes to employ in the study was utilized. As Hsieh and Shannon (2005) remarked qualitative content analysis has three distinct types which are conventional, directed, and summative. In this study, conventional content analysis was put to use. In conventional content analysis, researchers do not adhere to available categories in the literature instead, categories emerge from the collected data (Kondracki & Wellman, 2002). In this type of analysis, if the researcher collects the data through interviews, then the interviews need to be open-ended (Hsieh & Shannon, 2005), which requires the collected data to be read thoroughly to deduce codes, categories, and themes, respectively. Moreover, the collected studies were also analyzed with content analysis, and specifically, the ones that offer solutions to the identified problems were chosen. 3. Findings 3.1. The Views of Prep-Class Students on the Difficulties of Peer Evaluation In this part of the study, the first research question was answered by collecting the findings of the interviews under the relevant themes. 3.1.1. Students’ language ability and personality Peer evaluation requires students to criticize their classmates’ papers from various angles depending on the aim of the activity that the course teacher assigns. However, it was found that the personality and language ability of the students play a major role in their choice of evaluation. Most of the students, especially the ones that have low grammatical accuracy, and are not good at writing, found it difficult to evaluate their peers’ papers. As an example, one student stated the following: International Online Journal of Education and Teaching (IOJET) 2021, 8(2), 691-708. 695 Peer evaluation is a scary activity for me. I don’t like it! I don’t want to evaluate my friends’ papers not just because I find myself inadequate to do so, but also I don’t want my friends to criticize me. I prefer our course teacher to make comments on our papers. [P5] Generally, the students with avoidant personality indicated that they dislike peer evaluation as they care about “what my friend says to me if I criticize his/her paper.” These students mostly remarked that they do not want to evaluate their close friends’ papers or when they were asked to evaluate such kinds of papers, they said that they did not make so many comments. They just found some grammatical mistakes and made good comments on the overall integrity of the paper. One of the interviewees indicated that: When our course teacher says ok now exchange your paper and you are going to evaluate each other’s papers, I mostly try to stay away from the paper of my desk mate as I find it difficult to write comments under her gaze. Besides, I believe that she writes better than me and I don’t want her to see my ideas and my words. [P27] On the other hand, the ones that are good at writing in English and have confidence in themselves preferred peer evaluation, and they were eager to evaluate papers during the lesson. They sometimes even asked the teacher to do peer evaluation at the end of an in-class writing activity. Moreover, some confessed that they do peer evaluation outside the class when they are assigned writing homework. In this way, they said that they could check the mistakes in their papers before submitting them to the teacher. The two of such students stated during the interview: I really like peer evaluation in our writing course as I believe that it is a good activity to see what our friends are writing and to see each other’s’ mistakes as our writing teacher does. Of course, we might not find every mistake, and we generally focus on grammatical mistakes, I think it is a beneficial activity for us to learn from the mistakes of our classmates. [P4] I sometimes asked for peer evaluation to my friend when the teachers assigned us an essay to write outside the class. I wanted to check my paper in such cases and see if there is something missing that I could not see. I think peer evaluation is beneficial for us. [P9] Therefore, it can be said that students’ English level and ability in writing, as well as their personality, affect their choice of evaluation. Students with low ability in writing and an avoidant personality found peer evaluation activity as threatening and preferred teacher evaluation, while the ones that are good at writing and confident in their skills believed that peer evaluation is a beneficial activity for them to develop their writing skills. These students conducted peer evaluation even when the course teacher did not ask for it. 3.1.2. Having a clear purpose Another theme that emerged as a result of the interviews with the participants was that they had difficulty in understanding the purpose of the peer evaluation. This might result from the general educational system of the country where students generally depend on the teacher, and the views and critics of the teachers are highly valued. Previous courses that they got until tertiary education generally did not require them to evaluate each other’s papers instead, it was the responsibility of the course teacher to do so. Therefore, when asked during the interviews, Tuzcu-Eken 696 if they had ever evaluated their friends in previous courses that they got before, a majority of the students responded negatively. Two of the students commented: I really could not understand the gist of peer evaluation, even if our writing course teachers gave us some notes to follow during peer evaluation. I had never had such kind of evaluation when I was at high school so I believe that it is the teacher's responsibility to evaluate our papers. I don’t think that we are so good at evaluating each other’s papers as we are also learners. [P12] I found it difficult to evaluate our friends’ papers as I don’t think that I am good at writing, so how can I evaluate my friends. Yes, sometimes our writing teacher gives us worksheets that directs us on how to follow the evaluation process step by step, yet, I still believe that I am not good at seeing every detail and especially, it is hard to understand the shortcomings of the organization of the text. Grammar was a little bit easy though. [P35] It can be understood from the remarks of these students that even if the course teacher gave some instructions to the students to follow while they do peer evaluation, low-level students found the directions not enough, which left them behind in understanding the essence of the work done. It seems that those students still needed some guidance. On the other hand, the ones who have confidence in their writing and have high levels of English remarked that the peer evaluation activities were very beneficial for them, and they benefited from the activity not only for their own writing exercises but also for critical skills development. Such kind of students stated that: The instructions that the course teacher gave us during the peer evaluation activities were clear enough, and I could not have any difficulty in following the steps of the evaluation, and even if sometimes I had difficulty, I asked my teacher for an explanation. I believe that the teacher was clear enough to explain the reason behind such kind of an evaluation process, and if I become a teacher in the future, I will do the same. [P1] Most of the time I enjoyed evaluating my friends’ papers as I generally wonder what others write. I liked reading my friends' papers. The instructions of the teacher were clear enough, and I liked such kind of different activities that we followed in our writing course. I wish we did the same in other courses. [P3] Setting a clear purpose to the students, providing some explanations, and giving some instructions are important in encouraging students to evaluate their peers’ papers. However, it is understood that low-level students still need some more directions, and in this case, it is necessary for writing teachers to provide extra papers that steer such students in-detail. 3.1.3. Understanding the terminology and finding the relevant mistake Another issue that emerged, as a result, was that some of the students found it difficult to understand and learn the terminology of evaluation. These students had difficulty in learning the abbreviations such as G for Grammatical mistake, WO for a mistake in Word Order, or WT/WVT for Wrong Verb Tense. While course teachers evaluate students' papers they use such kinds of abbreviations to point out the mistakes that they see on the paper as it is hard to write the full form of the mistake every time. Generally, it is tough for the students to understand and learn these abbreviations, which require to be learnt in advance. In our case, participants indicated that even if the course teacher gave them the necessary terminology that they can use during peer evaluations, it was still hard for some to follow as International Online Journal of Education and Teaching (IOJET) 2021, 8(2), 691-708. 697 there are too many abbreviations, and it was time-consuming for them to check the abbreviation list repeatedly. Besides, they remarked that sometimes it was troublesome for them to understand in which category the mistake that they found belonged to. As some of the students stated: I sometimes found it difficult to categorize the mistake that I found even if I checked the notes that were given by the teacher. I could understand that there was a mistake there, but categorizing was impossible at times. [P22] It was occasionally time-consuming for me to categorize the mistake, and in such cases after trying a while, I remember myself ignoring the mistake as if I could not find it. [P27] Our course teacher asked us to show the category of the mistake instead of just correcting it. It was hard for me to find the category of the mistake. In fact, I prefer to correct it directly. [P15] The students that are good at grammar and writing, on the other hand, said that they did not have any obstacles while evaluating the papers. Two of them indicated that: It was easy for me to detect grammatical mistakes and tag it with the relevant terminology as the course teacher gave us a handout on which you can find the abbreviations of the mistakes. [P17] I did not have any problems identifying the grammar mistakes. It was easy with the handout given by the teacher. [P23] Besides, some of the mistakes were more difficult for students to identify than others. As stated by some: It was easy to find a spelling error or verb tense error, yet, I think finding the wrong word choice such as interested in or interested on was hard to me as I most of the time confuse such kinds of usages. [P18] Not every mistake was so easy. For example, I myself learnt that the word furniture was always single; however, it was difficult for me to detect it on my friends’ paper as I myself use it in the same way. [P37] The views of the students demonstrated that the terminology was a bit problematic for the students, and it needed to be handled carefully and in a detailed way by the course teacher. 3.1.4. Content and organization related mistakes In addition to the terminology and language-related mistakes, detecting the problems related to content and organization was a bit hard for the students. Content refers to the mistakes related to the appropriateness of the text to the topic and irrelevant sentences in the text. By organization, we mean the structure of the paper, such as the title, topic sentence, supporting ideas, concluding sentence, and their harmony. When compared to language-related mistakes, finding out the problems related to content and organization were more difficult for the students. Moreover, it was impossible to show all content and organization related mistakes with a handout. On this issue, some students said: Tuzcu-Eken 698 I sometimes had trouble understanding the unity of ideas. While reading my friend's paper, I see that there is something missing, yet, it was difficult for me to find what it is or to understand which sentence is irrelevant. [P30] It was hard for me to understand which sentence is unnecessary while reading the paper because sometimes all sentences looked as if they were wrong. [P35] I found it difficult to grasp the meaning of sentences, as there were many mistakes. They all seemed problematic in some cases. [P8] It was easy to find grammar mistakes, but it was sometimes difficult to understand the topic sentence, concluding sentence, etc. [P4] As it can be inferred from what students stated, finding the content and organization related mistakes were rough for all the students. It is also difficult for the teachers to show the students all related mistakes. Therefore, understanding such mistakes can inevitably take more time for the students. In that case, teachers need to provide lots of reading exercises for their students since reading supplies necessary knowledge for the writing skill. Besides, reading and writing courses can both be done in a way that each supports the other. 3.2. Recent Research on Peer Evaluation In this part of the study, the second research question was aimed to be answered by examining the relevant papers of recent research with an intention to find solutions to the problems experienced. 3.2.1. Online (computer-mediated) and anonymous evaluation In this day and age, the world revolves around technology, and so do the students of the modern era. Today it is impossible to keep learners away from technology, and there is not such an aim of modern language teaching. Teachers benefit from the technological products in their lessons, and students, in return, enjoy these types of technological applications, which create a more vivid and interactive learning environment. As for teaching the writing skill, some of the research supported the use of technology in and outside the classrooms (e.g., Li & Li, 2018) and found that online assessment had better gains than paper-based assessment (e.g., Hoomanfard, 2017; Huang, 2016; Li et al., 2020). The study of Hoomanford (2017), for instance, compared two groups of students, one of which applied conventional peer evaluation and the other computer-mediated peer evaluation. As a result, the researcher found that the ones who applied computer-mediated peer feedback were more content with the peer evaluation activities than the other group. The research of Li et al. (2020) also obtained similar results. The researchers concluded that computer-mediated peer assessment yielded better results than the paper-based peer evaluation. The findings of these studies can be a solution to the personality problems of the participant students of our study as they do not want to evaluate their peers publicly, or they stay away from making harsh judgements. Besides, anonymous peer evaluation might be a good alternative in this respect as students would not be in class, teachers can send the papers anonymously, and the name of the evaluator would stay anonymous as well. If possible, some programs like Turnitin can also be used as there is such an option of the program to assign the papers anonymously to the students in the same class for online peer evolution. Some of the investigated papers also suggest using blogs, WeChat platforms and Turnitin as an online peer evaluation (e.g., Huang, 2016; Li & Li, 2018; Ma, 2018) platform. The study of Huang (2016), for instance, revealed that both teachers and students indicated that blogs are beneficial in increasing the writing skills of students to some extent and are useful with some International Online Journal of Education and Teaching (IOJET) 2021, 8(2), 691-708. 699 components of feedback. Therefore, blogs can be used as an alternative to pen and paper peer evaluation, yet, as remarked by Huang (2016), they are not appropriate to all kinds of peer evaluation activities. For this reason, a balance between pen and paper evaluation and computer-mediated evaluation can be addressed. Li and Li (2018) investigated the usage of Turnitin in a writing course. The researchers addressed its benefits and limitations of using Turnitin for peer evaluation and indicated some features of it that may be beneficial in addressing the challenges identified in previous studies. As a result, the researchers remarked that Turnitin is beneficial in shifting students’ attention from local to global issues of writing, helping students provide more useful comments to their peers’ papers, and paving the way for classroom management in the course of peer evaluation. In another study, Ma (2018) recommended the use of WeChat platform by stating that such platforms stimulate students’ enthusiasm in writing and peer evaluation, and enable students to actively participate in the evaluation process more than teacher written feedback. To conclude, computer-mediated platforms may be beneficial for writing teachers who want to integrate students into the process of evaluation. In this way, avoidant and low-level students can attend the evaluation process easily and enjoy the benefits of peer review. 3.2.2. Training the raters beforehand In most of the studies investigated (e.g., Chong, 2017; Duruk, 2016; Meihami & Razmjoo, 2016) the basic problem of the students was a lack of assessment literacy. Therefore, in the literature, it is highly suggested to give training to the students beforehand, and some of the investigated studies did so (e.g., Duruk, 2016; Li et al, 2020). The study of Duruk (2016), for instance, revealed that in a writing class, students needed a pre-training to apply peer evaluation effectively, and although the ideas of the students were negative before the training, after the training, a positive change was found in the students' views against peer evaluation. The study of Chong (2017), on the other hand, investigated the relationship between the language ability of students and their accuracy in giving feedback to their peers. As a result, it was ascertained that there is a strong relationship between the ability levels of students and the accuracy and relevance of the feedback that they give. Accordingly, giving training to the raters can be a good choice when the ability of the students is considered. After training, as investigated in the literature, students’ ability in peer evaluation can increase and their ideas about peer evaluation may change. The study of Li et al. (2020) also supported the idea of giving training to the student evaluators. In this study, the researchers conducted an empirical study and divided the investigated group into two: one group did not participate in peer assessment, and the other group did peer assessment. As a result, they found that peer assessment had a positive effect on students learning, and the most effective factor is the rater training that was given to the group, which applied peer assessment. According to Nelson and Carson (2006) training gains more importance when the findings of the studies which revealed that students prefer teacher feedback to peer feedback is considered. In most of these studies, the reason behind the students’ preference was that students generally did not trust the evaluations of their peers (Guan & Su, 2016), and they mostly depended on the feedback of their teachers (Çalışkan & Kömür, 2019; Marsh, 2018). In order to overcome such kinds of prejudices of students in general, it is essential for writing teachers to indicate the necessity of peer evaluation in the writing development of students. Training students before assigning them peer evaluation duties would be useful for this cause. Giving training to the students can also be a solution to the problems that they experience with the terminology. The more they come across with similar problems, the more they get educated on how to detect such mistakes. The study of Carless and Bound (2018), for instance, investigated the development of student feedback literacy and focused on two learning Tuzcu-Eken 700 activities, one of which was analysing exemplars. Accordingly, exemplars were found to be beneficial in increasing the literacy of students as they provided rich examples of mistakes experienced by the students in similar situations. Therefore, in training designed by the course teacher, students need to come across many examples of mistakes with related terminology. In this way, they can extend their repertoire of mistakes as well as the terminology. Prior training can also increase the understanding of students on content and organization related issues on a paper by providing many examples during the training. In order to provide an effective training Anderson, Habbal and Bridges (2020) offer gameful learning pedagogy. In their study, the researchers developed a training for non-English students. They utilized two readings, a video, and several reflection questions in their training. In this training, the first reading aimed to introduce the best peer evaluation practices for students, the second reading assisted students on learning peer evaluation strategies, and the video introduced peer evaluation assignment rubrics that were meant to be used during peer evaluation activities. Emphasizing the importance of an efficient training, Guan and Su (2016) also suggest some training strategies which explain the purpose of peer evaluation, model the process, monitor each group, and evaluate the process for an effective peer evaluation. Irwin (2019), on the other hand, conducted a training through Moodle. In the first phase of the training, the researcher asked a list of questions to students regarding peer evaluation in order to raise awareness of students about the issue and develop some guidelines of peer evaluation together. Then, the researcher gave training through Moodle which modelled some paper evaluation exercises for students. As the researcher stated, Moodle was used specially to help students to revise the training when needed. To sum up, giving training to the students was found to be influential in both increasing the ability of students in peer evaluation and shaping their ideas about the effectiveness of peer evaluation. 3.2.3. Focusing one thing at a time with formative assessment While assigning peer evaluation activities to the students, formative peer assessment can be more beneficial in giving clear purposes to the students to focus on. Because in a summative assessment, students see the final product of their peers, and they are expected to focus on everything on the paper. Therefore, “summative assessment is challenging with students who may not feel comfortable marking the work of their peers” (Wride, 2017, p.4). However, during a formative assessment process, students get a chance to read their peers’ papers at certain intervals. By this means, they can concentrate on one thing at a time, which can provide clear purposes for them to focus on. The study of Khalim (2020), for instance, indicated that there are some appropriate strategies to follow while assigning peer evaluation tasks to the students, and one of these strategies is giving clear and detailed guidelines, which is possible with focusing on one thing at a time as suggested in formative assessment. Liu and Carless (2006) also remarked that students should not be asked to give grades as it will damage the nature of formative assessment. The outcomes of the peer evaluation should be on the development of the skills of the assessor as well as criticizing the paper being evaluated for the sake of giving constructive feedback to their peers about their writings. Wanner and Palmer (2018), on the other hand, argue that teachers need to be careful in organizing peer evaluation activities, and that in order the process to be successful, the capacities of students in giving feedback should be developed as well as giving importance to the inclusion of teachers to the process. In sum, formative peer evaluation can be beneficial when applied appropriately and designed in a way that it “support[s] students in providing feedback on substantive, content- https://www.tandfonline.com/doi/full/10.1080/07294360.2012.705262?casa_token=rQg7E175LRsAAAAA%3AOXqjyQk1DZtfAV9GBISvpRCZDuCTt_eh28u_8uGW173HH37TNX-d-btwnlaPOOYzk81t87s4okHjjQQ International Online Journal of Education and Teaching (IOJET) 2021, 8(2), 691-708. 701 related issues and in responding to such feedback” (Snowball & Mosterd, 2013). Summative assessment, on the other hand, can also be used at the end of the formative assessment process in order to assess the final products of the students. By combining both methods of peer evaluation, teachers can benefit from the various gains of each method. 3.2.4. The effect of culture on peer evaluation Culture which plays a role in foreign language learning and teaching also may have an influence on the process of peer evaluation in writing (Hu, 2002; Hyland & Hyland, 2006; Yu, Lee & Mak, 2016). As peer evaluation takes place in a cultural context, it is possible for learners to behave according to their cultural values during their evaluations (Lee, 2008). Moreover, several studies have revealed that cultural differences pose problems to peer evaluation, and students’ culture might prevent them from performing their best during peer evaluation (e.g. Carson & Nelson, 1996; Hyland, 2000). According to the findings of previous studies, heterogeneous student groups which are composed of students of various cultural backgrounds generally have different sociolinguistic peer interaction rules that might lead them to give useless feedback and cause misunderstandings and conflicts (Hyland, 2000; Hyland & Hyland, 2006). On the other hand, homogeneous student groups might be more successful in their interactions during peer evaluation (Nelson & Carson, 2006). However, there is a lack of empirical evidence to support these arguments. The principal cultural factors that are argued to affect students’ judgements during peer evaluation are “collectivism, power distance, the concept of face, and interpersonal harmony” (Yu & Lee, 2016, p. 477). Collectivism “emphasizes embeddedness of individuals in a larger group” (Gorodnichenko & Roland, 2012, p. 2), and collectivist cultures tend to prioritize group interests rather than individual interests. Power distance as stated by Hofstede (1985, p. 347) is “the extent to which the members of a society accept that power in institutions and organizations is distributed unequally”. Some cultures which have more inequalities in the society tend to have high power distance, while others which are more concerned with providing equality in the society or in groups have low power distance (Alper, 2019). The concept of face, on the other hand, is related to collectivist cultures and “refers to a person’s integrity, dignity, and self-respect” (Charoensuk, 2011, p. 158). Therefore, in collectivist cultures saving the face of a person is an important cultural behavior. Lastly, interpersonal harmony is again influential in collectivist cultures as it gives particular importance to harmony between groups of people. Accordingly, above given cultural aspects are regarded to have negative influence on students’ judgements as students of collectivist cultural backgrounds are found to refrain from making critical judgements on their peers’ papers (Hyland, 2000; Hu & Lam, 2010; Yu, Lee & Mak 2016). However, recent research has proved the exact opposite (e.g. Hu, 2019; Hu & Lam, 2010; Yu, Lee & Mak, 2016). For instance, the study of Yu, Lee and Mak (2016) investigated if Chinese homogeneous groups of students refrain from making judgements on each other’s papers during peer evaluation activities as stated by some research. The participants of the study were four Chinese male students having different learning experiences but belonging to the same culture, in a way, they are culturally homogeneous. During the study, the researchers examined three cultural issues: collectivism and group harmony, face-saving, and power distance. As a result, the study revealed that while these students had different views and beliefs, the cultural issues did not prevent the students from making critical judgements during peer evaluation. For this reason, the researchers suggested that previous studies might be wrong in their judgements about the hindrance of cultural issues during peer evaluation activities. Hu and Lam (2010) also examined cultural appropriateness and pedagogical effectiveness of peer evaluation for Chinese students since Chinese is accepted to have a collectivist culture. Consequently, the study indicated that the 20 Chinese participants of the study approved peer evaluation as a socioculturally appropriate pedagogical activity. Tuzcu-Eken 702 Nevertheless, the number of participants of these two studies was limited, so the findings of these studies cannot be generalized to other contexts. On the other hand, McLeay and Wesson (2014) compared Chinese and UK marketing students’ perceptions of peer evaluation. In this comparative study, the researchers found that there were cultural barriers hindering heterogeneous groups from making fair judgements during peer evaluation activities. Chinese students were reported to have a more positive approach while assessing their peers, and these students felt uncomfortable in giving poor marks to their peers. However, the students from the UK were found to be more critical in their evaluations. Another major finding of the study was that even if UK students could carry out more critical evaluations when needed, in fact both groups of students were not comfortable in giving grades to their peers. Therefore, this study indicated that different student groups might behave differently during peer evaluation, and culture, to some extent, might have a role in the evaluations of students. Moreover, even if some research proved the opposite, when compared to individualist societies, students in collectivist societies might have a tendency to comment more positively to their peers’ papers. Based on the controversy about the effect of culture on peer evaluation performance of students in the literature, Crowne (2020) investigated the impact of cultural variables on peer evaluation performance of students. As a result of this large-scale study, which included more than ten thousand students, it was ascertained that the impact of cultural variables on the evaluation performance of participants was weak, instead the most effective variable on the students’ performance was their English language ability. Since Turkey is accepted as a collectivist society such cultural issues might have an impact on the evaluations of students. However, to our knowledge, there is no study conducted in Turkey, which means that there is a lack of research and knowledge on the influence of cultural issues in a Turkish peer evaluation context. Consequently, since there is no consensus on the impact of culture on peer evaluation performance of students in a writing class, each context should be evaluated in its own right. For this reason, in our case, the student group was homogeneous, yet culture might have had an impact on the behaviour of these students, especially on students with avoidant personality. In that case, it is important to build face-saving peer evaluation activities which might be possible with anonymous peer evaluation activities. Besides, in such collectivist cultures teachers need to explain the benefits of peer evaluation in a detailed way since students from collectivist cultures tend to give only positive feedback to their peers not because they want to mislead them but they want to keep interpersonal harmony as they believe that negative comments might make their peers embarrassed (Charoensuk, 2011). Lastly, as power distance is important in collectivist cultures, students in a writing class generally prefer the course teacher to give feedback to their papers, and in such cases it is again the responsibility of the course teacher to show students the dual benefits of peer evaluation since peer evaluation activities are not only beneficial for the feedback receivers but also for the feedback givers (Berggren, 2015; Lundstrom & Baker, 2009). 4. Discussion and Conclusion In an educational environment, assessment is an indispensable part of teaching. Whatever the lesson is, there is a need to evaluate the progress of the students for many reasons, such as coming to a conclusion about the effectiveness of teaching or judging whether students need remedial teaching or not. In a writing course, as in the other courses, teachers are expected to evaluate and track the development of students, and there are many ways to evaluate the papers written in such a course. One of these is peer evaluation. Peer evaluation is beneficial in many International Online Journal of Education and Teaching (IOJET) 2021, 8(2), 691-708. 703 ways, and by assessing their peers’ papers, students can engage in contrasting their own papers to their peers’ papers that they are evaluating (Snowball & Mosterd, 2013). Moreover, while evaluating their peers’ papers, students have the chance to see different styles of writing as well as approaching a text from various ways such as noticing the language, content, and organizational structures on a paper. They also learn the relevant terminology, which is necessary to understand the teacher's feedback as well. However, it is also possible that less motivated students might not benefit from peer evaluation when compared to the more motivated ones (Smyth, 2004), especially the ones who are not so good at writing, and sometimes it is also possible for the most motivated students to get lost in the exercises and find the task daunting. The current study aimed to figure out the most influential ways of implementing peer evaluation in a writing course through querying the views of students on the difficulties that they experienced while doing peer evaluation in their writing course. As a result, to answer the first research question, four prevalent obstacles were attained through the interviews with the students. These are (a) lacking ability in language usage and having an avoidant personality; (b) not having clear purposes for evaluation; (c) lacking ability in dominating the relevant terminology, and (d) having difficulty in finding content and organization related mistakes. Relevant prospective papers were investigated in order to offer a solution to each of these difficulties that the students experienced. Therefore, each difficulty was intended to be explained with the corresponding indication found in the literature. As an answer to the second research question, the literature review ascertained that employing online (computer-mediated) and anonymous peer evaluation, giving prior training, focusing one thing at a time with formative assessment, and regarding the effect of culture on peer evaluation can play an important role in finding solutions to the problems regarding peer evaluation. The revealed difficulties experienced by the students as a result of the first research question and the solutions obtained from the literature review as a result of the second research question are presented comparatively below. First of all, it was found that students’ language ability plays a major role in their view of peer review. The students with low level of language did not like peer evaluation and preferred teacher feedback. The low-level students’ lack of ability in providing peer feedback can be handled by giving training to the students at the beginning of the course, as suggested by Duruk (2016) and Chong (2017). As the aim of the writing course is to enhance the writing skills of students, a lack of ability needs to be compensated with the necessary training to increase the gains of the course and prepare students for the peer evaluation process. In order to provide effective training, writing teachers might benefit from some example papers, videos, rubrics that identify necessary terminology and so on. The training should focus on modelling the evaluation process to the students, raising consciousness of the students about the benefits of peer evaluation, and letting them explore the process (Lam, 2010). The students who were good at writing found peer evaluation activities beneficial and applied peer evaluation not only for the specific assignments of the course teacher but also for the other papers that were not specifically asked by the teacher. The findings of Davies (2006) also support the findings of this study as he ascertained that better students are more eager to criticize their peers’ papers. However, it was also found that even students with better writing skills need peer evaluation training because most of them experience peer evaluation at tertiary education, and until then, they only get teacher evaluation. Besides, as it was revealed in this study, even students also had difficulty especially in content and organization related mistakes. Previous research indicated that training students on peer evaluation increases the success of evaluation (Liou & Peng, 2009). Tuzcu-Eken 704 Students’ personality was found to have an effect on their attitude to peer evaluation since some students had an avoidant personality and some others did not. To overcome the avoidant personality, anonymous and computer-mediated peer evaluation techniques can be applied, and in this way, the pressure that the students feel during evaluation can be lowered, which can also lead to a non-threatening environment that is also evident in previous research (e.g. Huang, 2016; Hoomanfard, 2017; Li et al., 2020). Computer-mediated peer evaluation also provides time to the students as it allows students to conduct peer evaluation out of the class. When students do peer evaluation during the lesson, because of the limited duration of the lesson, they generally do not have enough time to focus on the paper. Besides, culture might have an impact on avoidant behaviour. As the students of the present study belonged to a collectivist culture, they might have refrained from making harsh criticisms in order to maintain group integrity and interpersonal harmony. In order to reduce the effects of collectivism on peer evaluation practices of students, teachers might model example papers and explain the dual benefits of peer evaluation for students since peer evaluation provides benefits not only for the receivers but also for the givers. Moreover, teachers may model the language to be used for the evaluation. For example, instead of saying or writing “This sentence is wrong. You should not use this word here”, students may say/write “Check this sentence! You had better not use this word here because…”. In this way, students can save the face of their friends as well as protecting group integrity and interpersonal harmony. Another alternative might be anonymous evaluation, yet, when they meet in the class, students may ask each other who evaluated their paper or force the other students to say the evaluator of their papers. Therefore, it would be more beneficial if they learn the appropriate language which can also help them with the criticisms of the people outside the class. Since pre-training is significant and essential in solving nearly all the difficulties students experience, giving training before assigning the papers to the students for evaluation can also be a remedy to their lack of having a clear purpose. In such a training, teachers need to show the formative peer evaluation process to the students and give clear purposes for them to evaluate. For instance, in the first round, teachers can hand out the same paragraph which includes some language-related mistakes to each student and ask students to focus on language- related mistakes first. After they find these mistakes and discuss them together, students can be instructed to find content related mistakes such as if the text is appropriate to the topic and if there are any irrelevant sentences. Finally, they can continue their evaluations with organizational mistakes like coherence and cohesion. Focusing on one thing at a time can also decrease the possibility of students’ being lost in finding the purpose of the evaluation. When conducted together on an anonymous paper, peer evaluation training can enlighten the students on how to evaluate a paper. In such a collective learning environment, students can develop their higher-order thinking skills and increase their self-autonomy (Bostock, 2000). In this way, peer evaluation can be beneficial for students’ own papers as well. During the training, benefitting from the cyclical process of formative assessment can yield clear purposes for the students. In the end, they can also evaluate the final product with a summative assessment. The previous research indicated that students should not be asked to give grades; instead, they can just focus on giving comments on related issues (Davies, 2006; Falchikov, 2001). The difficulty in understanding the related terminology and their general problem in focusing on language-related mistakes rather than content and organization of the paper can be tackled by assigning one thing to focus at a time, which is also possible through formative assessment and training. Teachers can provide various exemplars in these training sessions as indicated above. Besides, as formative assessment requires students to intervene in the process of writing at relevant intervals, it can yield better dominance over the text that they are assigned to evaluate. International Online Journal of Education and Teaching (IOJET) 2021, 8(2), 691-708. 705 Difficulty in content and organization related mistakes may be handled by assigning some reading exercises to the students in which they focus on the content and organization of the text (topic sentence, supporting sentences, supporting ideas, concluding sentence etc.) and find cohesive devices that combine one sentence to the other to create cohesion. In this way, their awareness can be increased. Since it is not possible to show all content and organization related mistake examples to the students, raising their awareness about the parts of the texts can also show students how to explore a text and how to create a text of their own. Writing teachers may also cooperate with reading teachers and ask them to specifically focus on the organization and content of the texts that they read, and by this means, an integrity will be established between the two courses. In conclusion, this case study indicated that while assigning peer evaluation activities to the writing class students, teachers need to be aware of the challenges that students experience and find the necessary solutions to their problems such as giving training at the beginning of the term, modelling many examples during the trainings, assigning computer-mediated peer evaluation practices, benefitting from both formative and summative peer evaluation, and regarding the influence of ability, personality and culture of students on their peer evaluation behaviours and designing the training in this respect. Moreover, it is important to show the importance of peer evaluation not only for writing skill development but also for the development of all language related skills of students because in many cases, students are not aware of the benefits of peer evaluation. Overall, the findings of the current study are expected to be helpful to the teachers having the same problems in their writing classes. 5. Conflict of Interest The author declares that there is no conflict of interest. 6. Ethics Committee Approval The author confirms that the ethics committee approval of the study was obtained from Kırklareli University Scientific Research and Publication Ethics Board (Approval Number: 35523585-199-E.13417). Tuzcu-Eken 706 References Alqassab, M., & Panadero, E. (2020). Peer assessment. In S. Brookhart, et al. (Eds.), Routledge Encyclopedia of Education (pp. 1-13). New York: Routledge. Alper, S. (2019). Power distance. In V. Zeigler-Hill, T. Shackelford (Eds.), Encyclopedia of Personality and Individual Differences (pp. 52-76). Cham, Germany: Springer. Anderson, O. S., Habbal, N. E., & Bridges, D. (2020). A peer evaluation training results in high-quality feedback, as measured over time in nutritional sciences graduate students. Advances in Physiology Education, 44, 203-209. Berggren, J. (2015). Learning from giving feedback: A study of secondary-level students. ELT Journal, 69, 58–70. Bostock, S. J. (2000). Student peer assessment. Learning Technology. Retrieved from http://www.ilt.ac.uk/1072.asp Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315-1325. Carson, J., & Nelson, G. (1996). Chinese students’ perception of ESL peer response group interaction. Journal of Second Language Writing, 5(1), 1-19. Charoensuk, P. (2011). The relationship between an English writing classroom and Asian cultural issues when using peer feedback. Executive Journal, 31(3), 154-159. Chong, I. (2017). How students’ ability levels influence the relevance and accuracy of their feedback to peers: A case study. Assessing Writing, 31, 13-23. Crowne, K. A. (2020). Does national culture influence peer evaluations on global virtual teams? Journal of Teaching in International Business, 31(3), 191-213. Çalışkan, M., & Kömür, Ş. (2019). “Can I trust you? From students’ perspectives: How useful is peer feedback in writing classes? ILTERG Conference Proceedings (pp. 105-115), Antalya, Turkey. Davies, P. (2006). Peer assessment: Judging the quality of students work by comments rather than marks. Innovations in Education and Teaching International, 43(1), 69-82. Duruk, E. (2016). Deeper insights into university students' perceptions about peer feedback in second language writing: A phenomenological approach. Eğitimde Nitel Araştırmalar Dergisi (ENAD), 4(3), 28-42. Falchikov, N. (2001). Learning together: Peer tutoring in higher education. London: Routledge. Ferris, D. R., & Hedgcock, J. S. (2005). Teaching ESL composition: Purpose, process, and practice. Mahwah, NJ: Lawrence Erlbaum Associates. Gerring, J. (2004). What is a case study and what is it good for? The American Political Science Review, 98(2), 341-354. Gorodnichenko, Y., & Roland, G. (2012). Understanding the individualism-collectivism cleavage and its effects: Lessons from cultural psychology. In M. Aoki, T. Kuran, G. Roland (Eds.), Institutions and Comparative Economic Development. International Economic Association Series (pp. 213-236). London: Palgrave Macmillan. Guan, M., & Su, X. M. (2016). Study on training strategies for effective peer review. Cross- Cultural Communication, 12(12), 40-44. http://www.ilt.ac.uk/1072.asp International Online Journal of Education and Teaching (IOJET) 2021, 8(2), 691-708. 707 Hofstede, G. (1985). The interaction between national and organizational value systems. Journal of Management Studies, 22, 347–357. https://doi.org/10.1111/j.1467- 6486.1985.tb00001.x. Hoomanfard, M. H. (2017). EFL learners' attitudes and perceptions of online and conventional peer written feedback: A tertiary level experience. Malaysian Journal of Languages and Linguistics, 6(1), 49-62. Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288. Hu, G. (2002). Potential cultural resistance to pedagogical imports: The case of communicative language teaching in China. Language, Culture and Curriculum 15(2), 93-105. Hu, G., & Lam, S. T. E. (2010). Issues of cultural appropriateness and pedagogical efficacy: Exploring peer review in a second language writing class. Instructional Science, 38, 371– 394. Hu, G. (2019). Culture and peer feedback. In K. Hyland and F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (2nd ed., pp. 45-63). Cambridge, UK: Cambridge University Press. Huang, H. C. (2016). Students and the teacher’s perceptions on incorporating the blog task and peer feedback into EFL writing classes through blogs. English Language Teaching, 9(11), 38-47. Hyland, F. (2000). ESL writers and feedback: Giving more autonomy to students. Language Teaching Research 4(1), 33–54. Hyland, K., & Hyland, F. (2006). Feedback in second language writing: Context and issues. Cambridge: Cambridge University Press. Irwin, B. (2019). Enhancing peer feedback practices through screencasts in blended academic writing courses. The JALT CALL Journal, 15(1), 43-59. Khalim, A. (2020). Implementing peer assessment to improve the writing ability of the second year students of SMP Negeri 44 Muaro Jambi. Jurnal Ilmiah Universitas Batanghari Jambi, 20(1), 86-94. Kondracki, N. L., &Wellman, N. S. (2002). Content analysis: Review of methods and their applications in nutrition education. Journal of Nutrition Education and Behavior, 34, 224-230. Lam, R. (2010). A peer review training workshop: Coaching students to give and evaluate peer feedback. TESL Canada Journal, 27(2), 114-127. Lee, I. (2008). Student reaction to teacher feedback in two Hong Kong secondary classrooms. Journal of Second Language Writing, 17, 144-164. Li, J., & Li, M. (2018). Turnitin and peer review in ESL academic writing classrooms. Language Learning and Technology, 22(1), 27-41. Li, H., Xiong, Y., Hunter, C. W., Guo, X., & Tywoniw, R. (2020). Does peer assessment promote student learning? A meta-analysis. Assessment & Evaluation in Higher Education, 45(2), 193-211. Liou, H. C., & Peng, Z. Y. (2009). Training effects on computer-mediated peer review. System, 37(3), 514-525. https://doi.org/10.1111/j.1467-6486.1985.tb00001.x https://doi.org/10.1111/j.1467-6486.1985.tb00001.x Tuzcu-Eken 708 Liu, N.F., & Carless, D. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11(3): 279–290. Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the reviewer’s own writing. Journal of Second Language Writing, 18, 30–43. Ma, R. (2018). A study on the mode of peer revision in English writing based on WeChat platform. Advances in Social Science, Education and Humanities Research, 230, 145- 148. Marsh, V. L. (2018). Portal and gatekeeper: How peer feedback functions in a high school writing class. Research in the Teaching of English, 53(2), 149-172. McLeay, F., & Wesson, D. (2014). Chinese versus UK marketing students' perceptions of peer feedback and peer assessment. The International Journal of Management Education, 12(2), 142-150. Meihami, H., & Razmjoo, S. A. (2016). An emic perspective toward challenges and solutions of self- and peer-assessment in writing courses. Asian-Pacific Journal of Second and Foreign Language Education, 1(9), 1-20. Nelson, G., & Carson, J. (2006). Cultural issues in peer response: Revisiting “culture”. In K. Hyland & F. Hyland (Eds.), Feedback in Second Language Writing: Contexts and Issues (pp. 42-59). New York: Cambridge University Press. Panadero, E., Jonsson, A., & Alqassab, M. (2018). Providing formative peer feedback: What do we know? In A. A. Lipnevich & J. K. Smith (Eds.), The Cambridge Handbook of Instructional Feedback (pp.409-431). Cambridge: Cambridge University Press. Snowball, J. D., & Mostert, M. (2013). Dancing with the devil: Formative peer assessment and academic performance. Higher Education Research & Development, 32(4), 646-659. Song, Y., Hu Z., Guo, Y., & Gehringer, E. F. (2016). An experiment with separate formative and summative rubrics in educational peer assessment. IEEE Frontiers in Education Conference (pp. 1-7), Erie, PA, USA. Smyth, K. (2004). The benefits of students learning about critical evaluation rather than being summatively judged. Assessment & Evaluation in Higher Education, 29(3): 369–378. Swanborn, P. (2010). Case study research: What, why and how? London: Sage Publications. Wanner, T., & Palmer, E. (2018). Formative self- and peer assessment for improved student learning: The crucial factors of design, teacher participation and feedback. Assessment & Evaluation in Higher Education, 43(7), 1032-1047. Wride, M. (2017). Guide to peer assessment. Dublin: Trinity College. Yu, S., Lee, I., & Mak, P. (2016). Revisiting Chinese cultural issues in peer feedback in EFL writing: Insights from a multiple case study. Asia-Pacific Education Researcher, 25(2), 295-304. Yu, S., & Lee, I. (2016). Peer feedback in second language writing (2005-2014). Language Teaching, 49(1), 461-493. Zhao, H. (2018). Exploring tertiary English as a Foreign Language writing tutors’ perceptions of the appropriateness of peer assessment for writing. Assessment & Evaluation in Higher Education, 43(7), 1133-1145.