International Journal of Interactive Mobile Technologies(iJIM) – eISSN: 1865-7923 – Vol 16 No 09 (2022) Paper—Designing an Android-Based Peer Assessment E-Rubric Designing an Android-Based Peer Assessment E-Rubric https://doi.org/10.3991/ijim.v16i09.28107 Hilda Cahyani(), Ardian Wahyu Setiawan, Nadia Hanayeen English Department, Politeknik Negeri Malang, Malang, Indonesia hilda.cahyani@polinema.ac.id Abstract—Previous research has shown that peer assessment has benefi- cial effects on students’ learning. Inspired by the need to involve students in peer assessment for gaining constructive feedback for their works in translation and writing courses, we developed an android-based e-rubric. The study aims to create a standardized electronic rubric for facilitating self-and-peer assess- ments. Android was chosen as the assessment platform due to its customizability and affordability. Fifty-four students from two classes of English Department participated in this research. Interviews and focus-group discussions were conducted to identify the students’ needs and collect their feedback, while the online survey was done to measure their satisfaction on the developed applica- tion. The e-rubric was developed through eight stages: (1) needs analysis, (2) development of rubrics, (3) expert validation, (4) creating the e-rubric app, (5) testing of the rubric, (6) collecting users’ feedback, (7) product refinement, and (8) survey of users’ satisfaction. The results show that the developed e-rubric application supports collaborative works among the students. It allows them to perform peer-assessment more easily the developed e-rubric also gives them an engaging experience in completing the peer assessment. Students’ heterogeneity is the aspect in which teachers need to take into account in using the e-rubric application. This study has a focus on courses of productive skills i.e., translation and writing. The novelty of this research is using standardized electronic rubric for students to do self and peer assessment. Future researchers are expected to develop rubric assessment for receptive skills. Keywords—e-rubrics, android-based assessment, online teaching, peer assessment 1 Introduction Student involvement has a significant contribution to students’ learning. It enhances motivation, improves performance, reduces the feeling of isolation, and increases stu- dents’ satisfaction [26]. Involving students in meaningful learning, however, has been a challenge for teachers. In student-centered learning, teachers should create activi- ties focusing on students’ learning experience by employing collaborative work, team- work building, and cooperative learning [27] and [28]. Assessment, as one of important components of learning, has been acknowledged as providing benefits to students. iJIM ‒ Vol. 16, No. 09, 2022 191 https://doi.org/10.3991/ijim.v16i09.28107 mailto:hilda.cahyani@polinema.ac.id Paper—Designing an Android-Based Peer Assessment E-Rubric Much research has shown that peer assessment has beneficial effects on students’ learn- ing [29]. Peer assessment gives students opportunities to work collaboratively and pro- vide constructive feedback to their peers. In the Indonesian higher education system, vocational institutions generally adopt a curriculum oriented to providing more time for practices. Under the curriculum, stu- dents spend 60% of their study time for practices and 40% for learning concepts and theories in classroom. In both practice and theory sessions, students are required to work collaboratively and demonstrate cooperative learning. Peer assessment can be employed in both sessions to meet the learning demands. In some courses, such as translation and writing, students need to be provided with an opportunity to perform well. One of the instruments for doing assessment is rubric. Rubric is considered as an effective instrument for assessment since it provides consistency and fairness [16]. There has been some research developing rubrics for assessments of students’ com- petence, for example [4], and [14]. Analytical rubric has been found to better reveal students’ ability [4] whereas a meta-analysis of 75 studies by [14] reported that rubrics offered reliable scoring assessment to enhance student performance specifically for analytic, topic-specific and complemented rubrics with examples, and that it also poten- tially improves learning. However, as different classes/courses have specific charac- teristics, such as student level, needs, and materials, those rubrics cannot be directly adopted, especially for peer assessment. Generally, rubrics are paper-based, presented in a printed format. While the printed rubrics can be used for assessment purposes, they are less practical, thus requiring more time to obtain the results of assessment. Nowadays, teachers should not merely rely on face-to-face interactions in the classroom and utilize printed materials and tools. Tak- ing into account the demand of work-from-home, also study from home for students, we develop electronic rubrics which can be used outside the classroom and can be accessed by mobile devices, especially through the Android operating system. Android was favoured as the assessment platform due to its customizability and affordability, not to mention that android is the mostly used operating system in gadgets. Not much has been seen that peer assessment is a crucial aspect for supporting stu- dents’ learning development and leading to self-reflection boosting critical thinking skills. [6] reported that so far only 56% studies out of 36 papers used rubrics with students. Future research needs to investigate how students do the peer assessment to obtain a better understanding of the users and uses of rubrics. The purpose of this paper is to describe the design and implementation of an e-rubrics app for translation and writing courses during the pandemic to improve students’ criti- cal thinking. It provides insights into aspects which should be considered in the design and implementation of the e-rubrics. This article, therefore, addresses the following questions: • How is the android-based e-rubric designed to facilitate peer-assessment? • How is the e-rubric implemented in peer and self-assessment activities? • How did the students perceive/value the e-rubric? The contribution of this research is twofold. First, it provides insights into the devel- opment of an android-based application for peer assessment in translation and writing 192 http://www.i-jim.org Paper—Designing an Android-Based Peer Assessment E-Rubric courses based on students’ needs. Second, this research produces an android-based e-rubric application which becomes the practical contribution of the study. The rest of the article is organized as follows: the next section presents a review of relevant literature, followed by the methodology section which describes stages of the research. The next section outlines the results, covering the design of the e-rubric and the use of the application for peer assessment. The last section presents the conclusion of the study and offers recommendations for future research. 2 Literature review 2.1 Rubric assessment A rubric is a tool of assessment comprising some criteria along with the level of quality in each criterion [16]. It offers an objective-and-consistent evaluation to mini- mize the difference in grades even when multiple raters are involved [13]. Therefore, it is considered more reliable and consistent for assessments. In a rubric, we commonly find checklists using a simple yes/no or rating scales using a Likert scale all of which describe the performance for each criterion [6]. Commonly, it consists of two parts: the first is criteria expressing what to evaluate and the second is the description of perfor- mance level explaining the range of quality level from low to high [6]. 2.2 Types of rubrics [5] explained two types of rubrics which are commonly used by teachers, analytic vs. holistic; and general vs. task-specific. Analytic rubric presents “criteria one at a time” prioritizing feedback to students, whereas holistic rubric presents all criteria simultaneously but only needs one decision on one scale, usually used when students do not need feedback. General rubrics are used for similar tasks, such as maths requir- ing problem solving, or writing persuasive text. In this case, general rubrics are used to assess performance such as asking students to provide evidence and draw conclusions. Task specific rubrics, on the other hand, provide specific concepts and procedures that the students need to respond to. For example, in writing an essay, students need to clearly identify the textual evidence as well as what piece of conclusion they need to include in their writing. 2.3 Functions of rubrics Some research has reported that rubrics have the benefits of promoting learning and improving instructions [6, 24, 30, 31, 32]. Rubric can be personalized to make the assignment explicit and clear [6]. In order to get the maximum benefits from rubrics, developing appropriate criteria is important. The criteria of rubrics should be made based on learning goals and substantive appropriate performance level at which the quality of the work should be demonstrated [6]. iJIM ‒ Vol. 16, No. 09, 2022 193 Paper—Designing an Android-Based Peer Assessment E-Rubric [24] summarized from various sources [30, 31, 32] argue that a rubric must include an appropriate and optimal number of criteria, have operational criteria and perfor- mance-level descriptions, and contain informative performance-level descriptions. Referring to the principle of task-based language learning that “instruction needs to ensure that learners focus predominantly on meaning” [8], the use of assessment rubric can give students a meaningful experience in learning. Besides engaging students in the assess- ment task, rubrics also help students make a reflection of/reflect on their own learning. Assessment is basically conducted to identify students’ knowledge, ability and per- formance. Therefore, the use of rubrics in the classroom should accommodate both the experienced and novice users [3]. Therefore, it is necessary for rubrics designers to seek understanding of the rubrics’ uses and users. 2.4 Electronic rubrics Digital or electronic rubric has been widely used not only in education but also in other fields for various purposes along with the demand of technology and practicality. [7] proposes that electronic rubrics should contain features which facilitate feedback and reflection for students. This idea is crucial since students need to experience inter- activity/engagement when they do the peer assessment. This engagement is between them and the object of the assessment, in which they discuss with the peer whom the student assesses and finally process the assessment data and submit the data to their teacher. Electronic rubrics should be designed to promote fairness to reflect students’ per- formance. The challenge, however, is to raise students’ positive learning attitude and improve their professional skills [20]. 2.5 E-rubrics for assessment Generally, in conventional classrooms, assessment is conducted by the teacher for measuring students’ performance involving an ongoing process. Today, as students need to be involved in the assessment process, some methods such as “peer assess- ment and self- assessment” are chosen to stimulate students’ engagement since they afford multiple pieces of evidence to show the real performance [22]. Electronic rubrics are now considered more practical for this purpose. Some studies have shown the positive impact of assessment e-rubrics. Studies reported that e-rubrics are effective tools for performing formative and summative assessments [14, 18, 1, 15], promoting self-assessment [21], and measuring soft skills in Art subjects- both related to peer and self-evaluation [12]. E-rubrics are powerful resources for peer assessment [17]. Because of those advantages, it is interesting to study e-rubrics in terms of the design and uses in the classroom. 2.6 Studies of perception on the use apps Several studies have reported users’ perceptions of and attitudes toward the use of applications in everyday lives. [23] reports their study on parents’ perceptions on 194 http://www.i-jim.org Paper—Designing an Android-Based Peer Assessment E-Rubric educational apps used by their kindergarten children that the parents encouraged their children to use. Even though the results showed that parents’ attitudes and behaviour are varied, the research generally indicates that one-third of the participants (31.31%) had a mild-positive attitude whereas almost the same number participants showed a negative profile (29.07%) of against the educational app. Interestingly, parents’ edu- cational background and economic status are connected to how they monitored their children in using the app, showing students with lower economic status tended to play more games and spend more time watching TV. In other words, social status and eco- nomic background of the users can connect to the attitudes/behavior in which the app designers need to consider. Researchers have also investigated some elements that constitute an educational mobile application. One of the earliest researchers considered an innovator in mobile application evaluation is Walker [24]. His criteria have been used by other research- ers as a template. The criteria comprise curriculum, authenticity, and user friendliness. Some literature on this field in early 2010’s includes Mc.Manis and Parks in 2011 [24] who created 20 questions in Likert-scale format that inquired about if an app is edu- cational, age-appropriate, child-friendly, and pleasant to use. Vincent in [24] proposed these criteria for an educative mobile application. These are relevance, customization, feedback, thinking skills, participation, and sharing. Later researchers in the decade who investigated elements of a good educational mobile application include Lee and Cherner and Kucirkova in [24]. Lee and Cherner developed 24 dimensions of three categories: instruction, design, and engagement. Meanwhile, Kucirkova published a framework that encompasses five principles: col- laboration among teachers, researchers, and designers; shared epistemology; intercon- nected social factors; awareness of app affordances; and a child-centered pedagogy. 3 Methodology 3.1 ICT-based active methodologies This research follows a theoretical basis of ICT-based active methodologies [9] in which designing and creating ICT tool is aimed to optimize students’ learning. Another principle is also to construct the role of teachers as facilitators rather than instructors. The transformation for offline to online mode in this pandemic era made educators to be more flexible and open minded in learning, widening the opportunities for students to have more access to learning as well as to be more independent too. The designing of this e-rubric for students’ self-and-peer assessment led students to go beyond their social interaction among their peers, i.e. to construct their independent learning and critical thinking. In the traditional approach of teaching, teachers had a role to give tasks as well as marking the students’ work. Students relied on what teach- ers instructed to do. The concept of ICT-Based Active Methodologies brings a collaborative work among the researcher, IT specialist, students as well as the teachers. The transformative effects for students centered learning: iJIM ‒ Vol. 16, No. 09, 2022 195 Paper—Designing an Android-Based Peer Assessment E-Rubric • Improving interaction among students as peer works which promote collaborative works and discussion • More intense participation in the task work, doing more negotiation, communication and social activities • More acquisition for new resources and variety 3.2 Methods This research employs Design Based Research (DBR) methodology which focuses on understanding and documenting how and why designed innovations work in practice [11, 2]. Employing a collaborative approach that engages both researchers and practi- tioners, this study takes on an iterative process of systematically analyzing, designing, and evaluating the product [11]. DBR generates outputs deriving from the integration of theories and products based on particular needs. Data collection method consists of survey, semi-structured interviews, focus group discussion (FGD) and journals. The data collection comprises the following 8 stages: 1. Need analysis which was done in the beginning of the semester with some students who had taken translation, and writing courses and internships; 2. E-rubric development in which the rubric was developed based on the feedback obtained from the preliminary study; 3. Expert validation where the rubric was developed further based on feedback from experts; 4. E-rubric development; 5. The use of the e-rubric in the class; 6. Users’ evaluation and feedback; 7. Users’ survey; and 8. Product refinement. The preliminary study aimed to gain some information about why peer assessment was pivotal in translation and writing courses. Interviews and focus-group discussion were conducted to obtain data for needs analysis and users’ feedback, whereas the online survey was done to measure the students’ satisfaction. Translation and writing courses were selected since in these two courses, students are required to perform peer review as part of the learning process outlined in the course syllabus. This research involved two classes of English Department at a vocational higher education institution in East Java, comprising 54 students of second and fourth semes- ters taking Translation and Writing 1 courses. The participants of Translation were one class of 24 students, whereas Writing 1 comprised 30 students. Five students participated in a preliminary study in which they were interviewed. After the e-rubric was implemented in the classroom, ten students (five students from each class) were involved in the focus group discussion to explore their experience in using the rubric. Finally, all students of two classes took part in the survey of users’ satisfaction using Google Form. 196 http://www.i-jim.org Paper—Designing an Android-Based Peer Assessment E-Rubric The data from the interview were transcribed and closely analyzed to identify the themes and important information relevant to the research questions. The data of the interviews based on the themes are presented in the finding. The data of interviews and FGD were also summarized in a table to make sense of the presentation of the findings. 4 Findings This section reports on the stages in designing the android-based e-rubrics for stu- dents, how the rubrics are used for peer-and-self assessment, and the students’ percep- tion about the use of the rubrics. The electronic rubrics were created for students to facilitate self-and-peer assessment. 4.1 The design of e-rubrics The development of the rubrics comprised eight stages (see Figure 1): (1) needs analysis; (2) development of rubrics; (3) expert validation; (4) creating the e-rubric app; (5) the use of the rubric in the class; (6) collecting users’ feedback; (7) product refinement; and (8) survey of users’ satisfaction. 1. Needs analysis. The needs analysis was carried out at the beginning before learning began by conducting a Focus Group Discussion (FGD) with three students who have taken translation courses and carried out internship in the translation industry; 2. Development of the rubric. This phase was conducted based on the result of the needs analysis. The rubrics previously taken directly from some resources can be assembled and arranged according to user input; 3. Expert validation. The developed rubric was validated in a discussion involv- ing lecturers and researchers; 4. Development of the android-based rubric application. This phase involved profes- sional technology (IT) experts. The research team carried out the discussion with the IT experts and the application was updated 5 times, generating 5 versions of the app, which are: (1) debug-1.apk, (2) debug-5. Apk, (3) debug-8.apk, (4) rubric-16.apk, and (5) rubric-v1.0.2-2.apk; 5. The use of e-rubrics in learning. This phase includes several steps such as giving training to the students on how to use the rubric for peer assessment, and exploring their expectation after they completed the assessment (the detail of the implementation part is explained in the second part of the finding i.e, implementing rubric for peer assessment). 6. User evaluation. This phase was carried out online through FGDs with student representatives from the translation and writing courses. This phase generated feed- back from the experience and needs of the students, such as: adding a profile photo to the application, providing examples and detailed explanations for each criterion, providing criteria that only use one or two words to make the application concise and not too crowded, adding features “upload” (upload) for the text to be evaluated, etc. 7. Users’ survey. This phase resulted in feedback indicating that this application has efficiency in usability, offers ease of use, presents an attractive appearance, supports iJIM ‒ Vol. 16, No. 09, 2022 197 Paper—Designing an Android-Based Peer Assessment E-Rubric interaction with users, stimulates users to use this application (especially for com- pleting peer assessment tasks), and has updates. 8. Product refinement. The e-rubric app was improved based on the user satisfaction survey. The improvement includes: (a) adding a text box that allows users to pro- vide a descriptive evaluation, (b) adding brief information about the research, and (c) adding a “confirm” button to save the file before sharing. Fig. 1. Research stages in developing e-rubrics This research involves collaboration between lecturers, researchers, IT experts and students. The app was designed to be user-friendly with a simple appearance, easy to use, and on the Android platform. 4.2 Implementing e-rubrics for peer-assessment Using the e-rubric application in teaching and learning activities were another chal- lenge in this research. The students as the subject should be well-informed of why rubric was used as part of their learning and how they were expected to use it for peer assessment. This stage requires some prerequisite steps i.e.: 1. Providing training for students to use the electronic rubrics. This training was con- ducted at one meeting online because it was not possible to do the training in class or in person due to the pandemic. This is quite challenging because the application was new and had never been used previously; 2. Using the electronic rubric in the classroom. This electronic rubric was implemented in two classes, Translation and Writing 1, covering two learning topics. There were limitations to face-to-face conduct online: asking for extra time and effort from lec- turers to communicate with students via WhatsApp and LMS chats so that teaching and learning runs smoothly. In addition to that, lecturers must pair students with fair consideration and based on the effectiveness of learning. In this stage, students worked on their assignment individually then exchanged their work with their part- ner to conduct peer-assessment. The used the electronic rubrics to assess it; 198 http://www.i-jim.org Paper—Designing an Android-Based Peer Assessment E-Rubric 3. Conference. In this phase, the students negotiated and discussed the results of the assessment (electronic rubric results) with their peers. They explained the result to their partner and provided feedback to each other; 4. Self-assessment (reflection). After carrying out negotiations and discussions in the conference phase, each student performed a self-reflection in which they evaluated the work of the assessment results from their peers. They were asked to write down the reflection consisting of areas they were strong at and needed some improvement; 5. Collection of assessment results. The results of these assessments were recorded in the reports collected by the lecturers; 6. Final evaluation. The final evaluation was carried out by the lecturer to look at three elements in the students’ portfolio, namely the peer-assessment process, the conference, and the reflection. The objectives of looking at each element are to check whether students are able to edit their partner’s work, to check whether the conference where students are given the opportunity to discuss and negotiate feed- back has taken place, and to check whether students are aware of their strengths and weaknesses The six phases above indicate that the e-rubric development has gone through sev- eral stages in which the students were involved. This involvement was for the sake of learning where the electronic rubric has a function to support the students’ learning experience. 4.3 Students’ experience in using the rubric and performing peer-and-self assessment It is interesting to identify the students’ experiences in using the rubric as well as in implementing peer and self-assessment from the focus group discussion. One of the students stated that that the e-rubric application is versatile but still he found a technical problem as he mentioned in the excerpt below: So far, we did not encounter a significant problem with the application. How- ever, there was a time when I opened the app for the second time, it was “force- closed”. I had to make more trials to log in the app, then I succeeded. (Student 1) Student 1’s experience tells us that the improvement of the e-rubric application sys- tem is important to be done. To respond to this feedback, the researchers then worked with the IT expert to refine the app. Student 1 is one of critical students in the class, he also mentioned that he felt over- whelmed to work using multiple devices in doing this peer assessment. We have to use multiple devices in doing the peer assessment. First, we opened our peer’s work on the computer, we then evaluated and worked with the e-rubrics in our Android phone. I am thinking if we can upload our friend’s work in the app so we don’t need to use our computer. (Student 1) Some students found the rubrics were valuable to improve their critical thinking since they need to evaluate their friend’s work with some criteria and also finally give iJIM ‒ Vol. 16, No. 09, 2022 199 Paper—Designing an Android-Based Peer Assessment E-Rubric some comments in the descriptive box. Knowing some criteria of assessment stated in the rubric has made the students more aware about the assessment aspects. However, there was a problem where students felt confused to fit the case they found in their partner’s writing with the criteria in the rubric as mentioned below: I think this e-rubric is quite detailed. Previously, I only knew that our work was evaluated to be “minor” or “major” mistakes. In fact, this rubric has five criteria and this made me confused in making decisions, just like “should I give my peer 10 or 15?” I am really not sure. (Student 2) Since it was a new experience for the student in conducting peer assessment, it is considered normal to feel confused for fitting a score to their peer’s work. To cope with this situation, the teacher gave some models about how to score using this rubric by taking some examples from the students’ work. Students also experienced another issue of “unfairness” especially when they were scored lower than they deserved as mentioned by Student 3 below: I only made a minor mistake but my friend gave me a low score. I asked for an explanation but he could not provide me with a satisfactory answer. (Student 3) Student 3’s statement indicated that the assessor should be giving clear explanation to their peer’s work. Therefore, they should take this peer assessment as a serious work and finally should be able to satisfactorily explain their assessment results to the assesse satisfactorily. In essence, this peer assessment was a new experience for the students to learn how to basically improve their critical thinking, also seriously evaluate their friend’s work then finally communicate the assessment results to their peers. These stages gave a meaningful experience in taking part in learning. 4.4 Students’ perception of e-rubrics The results of the survey exploring students’ perception about e-rubrics comprise five characteristics: the usefulness of the e-rubric, the ease of use, the e-rubric’s look, the engagement/interaction between the user and the rubric, and the excitement in using the rubric as presented in Table 1: Table 1. Results of satisfaction survey of the e-rubric app Item Characteristics Strongly Disagree Disagree Agree Strongly Agree 1 Usefulness of the app (efficiency) – 4% 51% 45% 2 Ease of use (perspicuity) 2% 9% 36% 53% 3 The app’s look (attractiveness) – 13% 53% 34% 4 Interaction/engagement between the user and the rubric (dependability) 3% 17% 64% 16% 5 Excitement/motivation in using the app (stimulation) – 2% 53% 38% 6 Creativity of the app (novelty) – 6% 34% 60% 200 http://www.i-jim.org Paper—Designing an Android-Based Peer Assessment E-Rubric Table 1 reports that students gave positive feedback about the e-rubric in terms of interaction and engagement between the user and the rubric shown as the most pre- dominant characteristic (64% participants agree). Another feature of novelty of the app shows that the participants strongly agree that the e-rubric demonstrates creativity. Overall, the users were satisfied with the e-rubrics seen from their app’s perspective. Furthermore, we also present the data of feedback for project improvement (Table 2). Table 2. Feedback for project improvement No. Feedback from Students Yes No 1 Adding a feature of profile photo X 2 Giving an option to view examples in each criterion (can be correct or wrong examples) X 3 Adding text box where the users can give some descriptive evaluation  4 Adding the feature about (brief description about the research)  5 Adding confirm button (for saving the file)  6 Simple range of scoring for each criteria—uses one/two words/one phrase X 7 Adding the feature of upload text enabling the user to both see the rubric and the text simultaneously in one screen X 8 Placing the navigation button of “home” or “history” to the most reachable position to our thumb X 9 Giving two language options: English & Bahasa Indonesia to help those who do not really understand the English rubrics X Students gave substantial feedback for the improvement of the e-rubrics. Some important feedback is to add a text box so that the assessor can give descriptive com- ments, to give a brief description about the research (to inform users), and to add a confirmation button before the user saves the file. It is important to note that in creating this e-rubric, the researchers combined the assessment and the application principles. Both need to be included in the development of the e-rubric application. Not all students’ feedback can be taken into account/included (shown in a cross symbol) due to the space limitations and interface considerations. 5 Discussion This research employed some stages of designing the app which is an iterative pro- cess [13] requiring go-back-and-forth process and collaborative work of the research team with IT experts. This shows that developing e-rubric needs some consideration related to the content and also the use of the application. Furthermore, in designing the e-rubrics application, it is important to ponder some aspects of peer assessment; the design and use of rubrics for assessment; students’ learning needs (including their learning objectives); practicality and user-friendliness of android apps. This study corroborated some studies which proved that e-rubric has facilitated peer and self-assessment [17, 21]. In fact, there was a difficulty in running the training ses- sion for using the rubric with the student. The training was conducted using an online iJIM ‒ Vol. 16, No. 09, 2022 201 Paper—Designing an Android-Based Peer Assessment E-Rubric video meeting and as the pandemic had just begun at that time, we encountered some problems with the connection and interaction with students since they were not familiar with the synchronous platform at that time. As Brookhart [6] argues, designing rubrics should be based on learning objectives and students’ appropriate performance. This is obviously a challenge since we have to match the suitable criteria in the rubric and conduct a training for the students on how to perform the right assessment using the rubric. Even though the students stated their problems and difficulties in implementing this peer assessment, they actually built up a meaningful experience of learning [8] especially when they were involved in criti- cal thinking when doing the evaluation, also doing collaborative works or negotiation with their peers when they conducted the conference stage. This e-rubric can accom- modate an individual learning which involves analytical skill [10]. Even though it is time-consuming [22] taking some processes for students during their learning, and tak- ing part in the assessment gives them more learning experience. The students’ perceptions on the usefulness of the rubrics indicate that there should be an alignment between application and assessment purpose. This students’ perception corroborates with some studies which found that there were some important elements constituting the educational application, such as: a link-and-match between the appli- cation, assessment and curriculum, the engagement on user friendliness, and the aware- ness of application affordances [24, 25]. As the users of this e-rubric application are students, the application should be created based on their preferences and needs. The rubric is then expected not only to improve their understanding of the subject taught, but give students an opportunity to improve their analytical thinking and collaboration skills [7, 20]. 6 Conclusion and recommendation The developed e-rubric app is useful for collaborative works to assess each other’s strengths and weaknesses. By using this rubric for peer assessment, students can improve their experience in learning. However, teachers need to pay attention to the heterogeneity of the students. This is for facilitating equal contribution among them, especially for supporting the low-achievers to perform the correct peer assessment. The most important element in using the e-rubric of this study is designing the classroom activity in which students treasure their learning experience in peer assessment by using the e-rubric. Teachers have an important role to explain at the very beginning how to use the rubrics and how the rubric can facilitate students’ learning. The key results of this study are the e-rubric app supports collaborative working. With this app, students can perform peer assessment more easily. It also gives them engaging experience in completing peer assessment. The pedagogical contribution is that this study can contribute to attempts in understanding what we need to teach in writing and translation courses and help students develop editing skills in those courses. This study can also contribute to development of assessment rubrics for pro- ductive skills as well as practical courses. Last but not least, this study can help teachers develop instructional materials for writing and translation courses. 202 http://www.i-jim.org Paper—Designing an Android-Based Peer Assessment E-Rubric This e-rubric can be shared to other teachers and possibly will articulate feedback from them. This will lead to rubric co-creation and development through collaboration. Teachers tend to associate a set of rubrics to one assignment, but it is certainly possible to learn a great deal about one’s teaching methods and student learning across multi- ple assignments, and across multiple courses. The design and the implementation of e-rubric were only limited to the students’ taking courses of writing and translation. The e-rubric was designed for Android-based devices because most of our students used that operation system. Future research needs to be carried out to find a method or app to help students do peer and self-assessment for receptive skills, as in this study the application was developed for productive skills). Receptive skills are difficult to assess formatively since students do not produce tangible outcomes. So, it will be challenging to find a method/app to help them do peer assessment. Last, this app needs to be devel- oped in such a way that all platforms can use it. 7 Acknowledgment This study was financially supported by DIPA Politeknik Negeri Malang (SP DIPA- 023.18.2.677606/2020). We wish to express our deepest gratitude to all participants for contributing in this study. 8 References [1] S. Ahankari and A. Jadhav, “E-Rubrics: A Formative as Well as Summative Assessment Tool for Assessment of Course and Program Outcomes,” IEEE Eighth International Con- ference on Technology for Education (T4E), pp. 246–247, 2016. https://doi.org/10.1109/ T4E.2016.062 [2] T. Anderson and J. Shattuck, “Design-Based Research,” Educational Researcher, vol. 41, no. 1, pp. 16–25, 2012. https://doi.org/10.3102/0013189X11428813 [3] D. Bell, “Language Assessment: Principles and Classroom Practices (Third Edition): Review,” The Asian EFL Journal, vol. 24, no. 2, pp. 165–167, 2020. [4] N. Öztürk, M. Şahin, and M. İlhan, An Analysis of Scoring via Analytic Rubric and General Impression in Peer Assessment,” Turkish Journal of Education, vol. 8, no. 4, pp. 258–275, 2019. https://doi.org/10.19128/turje.609073 [5] S. Brookhart and A. Nitko, “Educational Assessment of Students: 8th Edition,” Pearson Higher Ed, 2018. [6] S. Brookhart, “Appropriate Criteria: Key to Effective Rubrics,” Frontiers in Education, vol. 3, no. 22, 2018. https://doi.org/10.3389/feduc.2018.00022 [7] A. Campbell, “Application of ICT and Rubrics to the Assessment Process Where Professional Judgement is Involved: Features of an e-Marking Tool,” Assessment & Evaluation in Higher Education, vol. 30, no. 5, pp. 529–537, 2005. https://doi.org/10.1080/02602930500187055 [8] R. Ellis, “Principles of Instructed Language Learning,” The Asian EFL Journal, vol. 7, no. 3, 2005. https://doi.org/10.1016/j.system.2004.12.006 [9] V. Gámiz-Sánchez, “ICT-Based Active Methodologies,” Procedia Social and Behavioral Science, vol. 237, pp. 606–612, 2017. https://doi.org/10.1016/j.sbspro.2017.02.018 [10] R. Carmen-Pamittan and C. Malenab-Temporal, “Language Pedagogical Styles in Technical- Vocational Education,” The Asian ESP Journal, vol. 52, 2018. iJIM ‒ Vol. 16, No. 09, 2022 203 https://doi.org/10.1109/T4E.2016.062 https://doi.org/10.1109/T4E.2016.062 https://doi.org/10.3102/0013189X11428813 https://doi.org/10.19128/turje.609073 https://doi.org/10.3389/feduc.2018.00022 https://doi.org/10.1080/02602930500187055 https://doi.org/10.1016/j.system.2004.12.006 https://doi.org/10.1016/j.sbspro.2017.02.018 Paper—Designing an Android-Based Peer Assessment E-Rubric [11] C. Ford, D. McNally, and K. Ford, “Using Design-Based Research in Higher Education Innovation,” Online Learning, vol. 21, no. 3, 2017. https://doi.org/10.24059/olj.v21i3.1232 [12] N. Haro-Garcia, M. Comas-Lopez, K. Hincz, M. Mazalu, and G. Sacha, “Soft Skills Assess- ment in Art and Globalization,” In Proc. The Sixth International Conference on Technologi- cal Ecosystems for Enhancing Multiculturality, pp. 199–204, 2018. https://doi.org/10.3390/ app11219923 [13] M. Huba and J. Freed, “Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning,” Allyn & Bacon, 160 Gould St., Needham Heights, MA 02494, 2000. [14] A. Jonsson and G. Svingby, “The Use of Scoring Rubrics: Reliability, Validity and Edu- cational Consequences,” Educational Research Review, vol. 2, no. 2, pp. 130–144, 2007. https://doi.org/10.1016/j.edurev.2007.05.002 [15] J. Otey, M. Agost, M. Contero, and J. Camba, “Teachers as Designers of Formative E-rubrics: A Case Study on the Introduction and Validation of Go/No-Go Criteria,” Uni- versal Access in the Information Society, vol. 18, no. 3, pp. 675–688, 2019. https://doi. org/10.1007/s10209-019-00686-7 [16] K. Ragupath and A. Lee, “Beyond Fairness and Consistency in Grading: The Role of Rubrics in Higher Education,” Diversity and Inclusion in Global Higher Education, pp. 73–95. Palgrave Macmillan, Singapore, 2020. https://doi.org/10.1007/978-981-15-1628-3_3 [17] M. Raposo-Rivas and M. de la Serna, “Technology to Improve the Assessment of Learning,” Digital Education Review, vol. 35, 2019. https://doi.org/10.1344/der.2019.35.%25p [18] Y. Reddy and H. Andrade, “Review of Rubric Use in Higher Education,” Assess- ment & Evaluation in Higher Education, vol. 35, no. 4, pp. 435–448, 2010. https://doi. org/10.1080/02602930902862859 [19] P. Sadler and E. Good, “The Impact of Self- and Peer-Grading on Student Learning,” Educa- tional Assessment, vol. 11, no. 1, pp. 1–31, 2006. https://doi.org/10.1207/s15326977ea1101_1 [20] D. Sadler, “Transforming Holistic Assessment and Grading into a Vehicle for Complex Learn- ing,” Assessment, Learning and Judgement in Higher Education, edited by Gordon Joughin (Dordrecht: Springer), pp. 1–19, 2009. https://doi.org/10.1007/978-1-4020-8905-3_4 [21] K. Steffens, “E-rubrics to Facilitate Self-Regulated Learning. REDU. Revista de Docencia Universitaria, vol. 12, no. 1, pp. 11–12, 2014. https://doi.org/10.4995/redu.2014.6417 [22] N. Van Loi, “Vietnamese EFL Teachers’ Beliefs and Practices of Alternative Assessment in Teaching English at Secondary School,” Asian EFL Journal, vol. 24, no. 2, 2020. https://doi. org/10.5746/LEiA/LA_Asia [23] J. Vaiopoulou, S. Papadakis, E. Sifaki, D. Stamovlasis, and M. Kalogiannakis, “Parents’ Perceptions of Educational Apps Use for Kindergarten Children: Development and Vali- dation of a New Instrument (PEAU-p) and Exploration of Parents’ Profiles,” Behavioral Sciences, vol. 11, no. 6, p. 82, 2021. https://doi.org/10.3390/bs11060082 [24] S. Papadakis, M. Kalogiannakis,and N. Zaranis, “Designing and Creating an Educational App Rubric for Preschool Teachers,” Education and Information Technologies, vol. 22, no. 6, pp. 3147–3165, 2017. https://doi.org/10.1007/s10639-017-9579-0 [25] S. Papadakis, J. Vaiopoulou, M. Kalogiannakis, and D. Stamovlasis, “Developing and Exploring an Evaluation Tool for Educational Apps (E.T.E.A.) Targeting Kindergarten Children,” Sustainability, vol. 12, no. 10, p. 4201, 2020. https://doi.org/10.3390/su12104201 [26] F. Marti and D. Bolliger, “Engagement Matters: Student Perceptions on the Importance of Engagement Strategies in the Online Learning Environment,” Online Learning, vol. 22, no. 1, pp. 205–222, 2018. https://doi.org/10.24059/olj.v22i1.1092 [27] P. Rao, “Collaborative Learning in English Language Learning Environment,” Research Journal of English Language and Literature (RJELAL), vol. 7, no. 1, pp. 330–339, 2019. https://doi.org/10.33329/rjelal.7119.330 204 http://www.i-jim.org https://doi.org/10.24059/olj.v21i3.1232 https://doi.org/10.3390/app11219923 https://doi.org/10.3390/app11219923 https://doi.org/10.1016/j.edurev.2007.05.002 https://doi.org/10.1007/s10209-019-00686-7 https://doi.org/10.1007/s10209-019-00686-7 https://doi.org/10.1007/978-981-15-1628-3_3 https://doi.org/10.1344/der.2019.35.%25p https://doi.org/10.1080/02602930902862859 https://doi.org/10.1080/02602930902862859 https://doi.org/10.1207/s15326977ea1101_1 https://doi.org/10.1007/978-1-4020-8905-3_4 https://doi.org/10.4995/redu.2014.6417 https://doi.org/10.5746/LEiA/LA_Asia https://doi.org/10.5746/LEiA/LA_Asia https://doi.org/10.3390/bs11060082 https://doi.org/10.1007/s10639-017-9579-0 https://doi.org/10.3390/su12104201 https://doi.org/10.24059/olj.v22i1.1092 https://doi.org/10.33329/rjelal.7119.330 Paper—Designing an Android-Based Peer Assessment E-Rubric [28] A. Rubio and E. Moore, “Group Work in the English Classroom: Cooperative and Collabo- rative Learning,” 2019. [29] C. Adachi, J. Tai, and P. Dawson, “Academics’ Perceptions of the Benefits and Challenges of Self and Peer Assessment in Higher Education,” Assessment & Evaluation in Higher Educa- tion, vol. 43, no. 2, pp. 294–306, 2018. https://doi.org/10.1080/02602938.2017.1339775 [30] D. Allen and K. Tanner, “Rubrics: Tools for Making Learning Goals and Evaluation Criteria Explicit for Both Teachers and Learners,” CBE-Life Science Education, vol. 5, no. 3, pp. 197–203, 2006. https://doi.org/10.1187/cbe.06-06-0168 [31] B. Moskal, “Scoring Rubrics: What, When and How?” Practical Assessment, Research, and Evaluation, vol. 7, no. 3, 2000. https://doi.org/10.7275/a5vq-7q66 [32] M. Roblyer and W. Wiencke, “Design and Use of a Rubric to Assess and Encourage Interac- tive Qualities in Distance Courses,” American Journal of Distance Education, vol. 17, no. 2, pp. 77–98, 2003. https://doi.org/10.1207/S15389286AJDE1702_2 9 Authors Hilda Cahyani is a lecturer at the English Department, Politeknik Negeri Malang, Indonesia. She completed her Ph.D. in TESOL at the University of South Australia. Her research interests are code-switching and digitalization in ELT. (e-mail: hilda. cahyani@polinema.ac.id). Ardian Wahyu Setiawan is a lecturer at English Department, Politeknik Negeri Malang, Indonesia. He obtained his Doctor of Education from the University of Adelaide and Master of Education from Monash University, Australia. His works revolve around teachers’ professional identity, ESP, MALL and CALL. Nadia Hanayeen is a lecturer at the English Department, Politeknik Negeri Malang, Indonesia. She completed her M.Ed. in TESOL at the University of Arkansas. Her research interests are formative assessment and translation pedagogy. Article submitted 2021-11-06. Resubmitted 2022-03-07. Final acceptance 2022-03-22. Final version published as submitted by the authors. iJIM ‒ Vol. 16, No. 09, 2022 205 https://doi.org/10.1080/02602938.2017.1339775 https://doi.org/10.1187/cbe.06-06-0168 https://doi.org/10.7275/a5vq-7q66 https://doi.org/10.1207/S15389286AJDE1702_2 mailto:hilda.cahyani@polinema.ac.id mailto:hilda.cahyani@polinema.ac.id