Australasian Journal of Educational Technology, 2017, 33(5). 15 Wikis for group work: Encouraging transparency, benchmarking, and feedback Amir Abdekhodaee Swinburne University of Technology Anne-Marie Chase Australian Council for Educational Research Bella Ross Monash University Technology is recognised as playing a part in the changing landscape in higher education; altering delivery modes and providing flexible opportunities for learning. Research into the use of wikis has shown that they provide many opportunities for student learning and the development of twenty-first century skills, however, there has been limited success in their use for collaboration. In this exploratory research, we report on a group wiki project in an engineering management unit at an Australian university. A wiki was introduced to replace the existing group report assessment to add transparency to the task. Each group had their own wiki and students were required to provide feedback to other group wikis which enabled transparency of students’ report development and group progress. The research aim was to investigate student perceptions of using a wiki. Students found the wikis helpful for benchmarking their activity against their peers. Students stated that peer feedback had improved their work; however, much of the feedback given by peers was too brief to be constructive, and provided too late to be useful in guiding their work. This paper will be of interest to academics interested in using wikis in their teaching to develop feedback, transparency, and benchmarking. Introduction A wiki is an online page that can be edited by a number of people. When edits are made, the wiki saves the newer version. Wikis allow multiple users to work together on a project, synchronously and asynchronously, and from any location so long as there is internet access. Using a wiki facilitates both task collaboration and task delegation. Collaboration involves students working together on a task and is more than simple task delegation. While task delegation sees students work in isolation, collaborative learning involves students working together to complete a task or solve a problem. Task delegation sees students divide work tasks then separately complete each task before combining their work in a single place. Using an online platform to work collaboratively on projects may reflect an emerging employment need (Putro, Carbone, & Sheard, 2014; Zorn, Sasaki, Qian, & Chase, 2016). Using a wiki to facilitate group work may provide opportunities for students’ learning by providing opportunities for collaborative learning (Blumenfeld, Marx, Soloway, & Krajcik, 1996; Caple & Bogle, 2013; Zorko, 2009), knowledge sharing and construction (Elgort, Smith, & Toland, 2008), and opportunities to develop interpersonal and negotiating skills. The ability to work in a team environment is one of several skills considered necessary for employability, and is reflected in graduate attributes and calls by industry to prepare students for their working life (Kalfa & Taksa, 2015; Nagarajan & Edwards, 2014; Thomas & Day, 2014). A wiki was introduced to a unit, based on previous observations by the first author that group work often resulted in group conflicts, and poorly structured and overlapping work of varying quality. The aim of this research was to investigate student perceptions of the impacts of using a wiki for a group work assignment. The research objectives were to examine how students perceived using a wiki impacted the group work process in terms of the transparency of work contributions, the benchmarking of work quality, and the provision of feedback on peers’ work. This research reports on survey and focus group data gathered from a third year engineering unit at an Australian university in semester 1, 2015. Australasian Journal of Educational Technology, 2017, 33(5). 16 Literature review: Using wikis in higher education Wikis allow individual contributions to be traceable thereby increasing individual accountability in group work situations (Caple & Bogle, 2013). Wikis can furthermore be monitored continually by a teacher and peers to determine progress by both groups and individuals. This can benefit feedback practices throughout the course of a project. Students are generally positive towards using a wiki for their studies (Baltzersen, 2010; Caple & Bogle, 2013; Clougherty & Wells, 2008; Kear, Woodthorpe, Robertson, & Hutchison, 2010; Lin & Kelsey, 2009; Nejkovic & Tosic, 2014; O'Connor, 2010; Popescu, 2014; Zorko, 2009). Not all of the research into wikis is positive. Many studies have found that student contributions to wikis often lack meaningful content and collaboration (De Wever, Van Keer, Schellens, & Valcke, 2011; Hadjerrouit, 2012; Judd, Kennedy, & Cropper, 2010; Karasavvidis, 2010; Lin & Kelsey, 2009): Students made simple edits considerably more often than they provided comments, suggesting that students were focused on meeting the wiki contribution requirements rather than collaborating to meaningfully discuss content. A minority of students contribute the most content on wikis (Carr, Morrison, Cox, & Deacon, 2007; Judd et al., 2010; Popescu, 2014; Sampaio‐Maia, Maia, Leitão, Amaral, & Vieira‐Marques, 2014). Despite the transparency of a wiki, students do not always contribute equally in group work situations (Carr et al., 2007; Judd et al., 2010; Neumann & Hood, 2009; Popescu, 2014; Robertson, 2008; Sampaio‐Maia et al., 2014). One study, for example, showed that 90% of all wiki actions were performed by approximately half of the students and that 20% of students performed at least half of all the actions on the wiki (Popescu, 2014). Students tend to hand in the bulk of their work closest to the deadline (Forte & Bruckman, 2006; Hadjerrouit, 2012; Judd et al., 2010; Sampaio‐Maia et al., 2014) and many contribute to the wiki on a single day only (Judd et al., 2010). This makes meaningful collaboration difficult if not impossible. Several studies have shown that students avoid contributing to wikis unless they are required to do so. One study (Cole, 2009) used a wiki in a course as an additional activity and did not have a single student use it out of the 75 students enrolled. Another study had none of the 287 students create or edit a wiki page over an entire semester (Ebner, Kickmeier-Rust, & Holzinger, 2008). The time commitment required is a common reason cited for students not contributing to a wiki (Cole, 2009; Karasavvidis, 2010; Popescu, 2014). Many students do not feel comfortable editing their peers’ work and may be reluctant to do so (Allwardt, 2011; Baltzersen, 2010; Cunningham, O'Donoghue, & Jennings, 2016; Dalke, Cassidy, Grobstein, & Blank, 2007; Foley & Chang, 2008; Grant, 2009; Ioannou & Artino, 2009; Kear et al., 2010; Lin & Kelsey, 2009; Lund, 2008), particularly without first discussing it with the group (Neumann & Hood, 2009; Robertson, 2008). Others do not feel comfortable having their work edited and critiqued by their peers (Karasavvidis, 2010). Technical issues in conjunction with using wikis were also commonly cited by students as a negative (Neumann & Hood, 2009; Robertson, 2008). Furthermore, some students do not find wikis particularly interesting in themselves or consider that they negatively impact on group collaboration (Cole, 2009; Elgort et al., 2008; Witney & Smallbone, 2011). Many studies suggest that wikis are not inherently collaborative and that further guidance or incentives are needed to promote meaningful collaboration among students (Judd et al., 2010; Zorko, 2009). Student collaborations in a wiki are likely to be ineffective if they lack appropriate learning design, pedagogical support, and scaffolding (De Wever, Hämäläinen, Voet, & Gielen, 2015; Foley & Chang, 2008; Judd et al., 2010; Larusson & Alterman, 2009; Lin & Kelsey, 2009; Zheng, Niiya, & Warschauer, 2015). Here scaffolding refers to the strategies and devices that support student learning (Rosenshine & Meister, 1992; van Merriënboer, Kirschner, & Kester, 2003). Group work at university is largely viewed in a negative light (Caple & Bogle, 2013) with the assessment of group work a major concern to students. Challenges include whether marks are to be distributed equally or given individually, and how individual student contributions are weighted (Caple & Bogle, 2013). A study aimed at using wikis to resolve some of these issues with group work (Caple & Bogle, 2013) found that students were as frustrated by the group work using wikis as other formats. In terms of transparency and fairness however, students were very positive. Students reported that knowing that all contributions were tracked meant that they did not waste time trying to get absent members to contribute, as they knew they would be caught out. Students also report that they appreciate being able to identify individual contributions by group members (Osman-Schlegel, Fluker, & Cheng, 2011). Teachers report that using a Australasian Journal of Educational Technology, 2017, 33(5). 17 wiki makes managing and assessing group work easier and more effective, as they can judge both the quantity and quality of contributions by individual group members (Elgort et al., 2008; Zheng et al., 2015; Zorko, 2009). The transparency of wikis may increase students’ motivation to improve their performance when they know that their work will be viewed by others (Dalsgaard & Paulsen, 2009); peers, teachers (O'Connor, 2010), and the general public (Zorko, 2009). Furthermore, the fact that peers can view, edit, and benefit from wiki work is a motivating factor for students (Baltzersen, 2010). Wikis can be used by students to benchmark their activity and work against their peers’ to raise their achievement (O'Connor, 2010; Popescu, 2014). Students learn from each other, can check whether they are on track by comparing their work to that of peers, and get ideas from other groups’ work (Popescu, 2014; Zorko, 2009). This increases student motivation to perform well (Popescu, 2014) and stand out from the other groups. Wikis can be monitored continually by a teacher and peers to see the progress made by individuals and groups. Continual feedback can benefit the overall feedback practices throughout the course of a project or a semester. A study of a wiki used for assessment purposes (Zorko, 2009) reported that the feedback process was beneficial for students. Students collaborated on documents and provided feedback on each other’s work prior to uploading it to the wiki. The teacher also provided prompt feedback to students’ queries, comments, and work. Another study of a wiki used in a writing class found that approximately 80% of students used the comments and feedback provided by their peers to further refine their writing (Forte & Bruckman, 2006). In a report writing wiki task, students commented positively on the benefits of receiving feedback from their peers (Neumann & Hood, 2009). Methodology In this study, we report on our initial findings from a group wiki project in an engineering management unit at an Australian university. A wiki was introduced to replace the existing group report assessment to add transparency to group project management activities and report writing. Participants and context This research focuses on a wiki group work project conducted in semester 1, 2015. In groups of three or four (and in some cases two), students were required to analyse an organisation and then report and present on the effectiveness of its managerial decisions. The group size of three to four was determined based on previous experience managing the task and the projected workload requirements of the task. Each group had their own wiki which could be viewed by the entire cohort. Students were required to provide three separate sets of feedback to another group’s wiki during the semester. This assessment was created to increase students’ understanding of management concepts by linking theoretical concepts to practical reports for different organisations. The wiki component of the assessment was worth 26% of the overall mark. The group work assessment was initially converted into a wiki task in semester 1, 2011. This was done in response to the unit convenor’s finding that: • project reports were of poor quality, • project reports were poorly structured and incoherent with overlapping material, and • some groups were experiencing internal conflict regarding tasks, workload, and timing. For these reasons, the project report assessment was redesigned to take the focus off the outcome of the report itself and instead place emphasis on providing opportunities for peer feedback – particularly for students who needed it the most. The project was divided into stages where peer feedback would be given three times during the semester. This increased the feedback and interactions between students and the teaching team. The wiki assessment has been used in nine iterations since it was first introduced into the unit. The engineering management unit at Swinburne University of Technology is a third-year compulsory unit for engineering undergraduate students form mechanical, civil, electrical, product design, biomedical, and software and telecommunication engineering. The unit has face-to-face lectures and tutorials and has been Australasian Journal of Educational Technology, 2017, 33(5). 18 designed to contain some fundamental management and project management concepts. Its aim is to teach students about the principles of planning, organisation, leading, control, and organisational behaviour. Students who take the unit represent diverse groups. Some work part time, some are mature-aged students, and some are international students. In addition to this diversity, student aptitudes vary considerably. Data collection and analysis This case study investigates student perceptions of a wiki in an undergraduate engineering unit using both qualitative and quantitative methods. We report on results from student surveys and focus groups for a student cohort of 286 students in semester 1, 2015: • student surveys: 82 students, and • student focus groups: 10 students across two sessions. Students were invited to participate in the research through announcements on the learning management site for the unit and in lectures. An announcement was posted on the learning management system in week 10 of semester, to inform students about upcoming surveys of student perceptions of using a wiki. Given that generality (external validity) was important to this study, the selection of the sample from which the data was collected aimed to provide maximum variation. Students were informed that their responses were anonymous. The surveys were distributed and collected in hardcopy format in lectures and tutorials during week 12 of the semester. The survey comprised 15 questions, which included 5-point Likert scale and open- ended questions. The results were transferred into the online survey platform Opinio for analysis. Of the 286 students enrolled in the unit in semester 1, 2015, 82 students completed the anonymous survey. This gives a response rate of 29%. By determining the sample was representative, it was possible to generalise the results to the population, therefore confirming external validity. In lectures and through announcements on the learning management site, students were invited to volunteer to participate in focus groups after the completion of the teaching semester and after they had received their final marks. Students were informed that their responses would be anonymous. There were two focus groups, each comprising five students and each lasting approximately 50 minutes. The focus group sessions were audio recorded and transcribed. Students were reimbursed for their participation with a $50 Coles/Myer voucher. The focus group questions explored the themes of transparency, benchmarking, and feedback. The analysis here follows the mixed-methods approach (Bazeley, 2012) where several complementary data sources are integrated for analysis. In this study, the data comprise student surveys and student focus groups. Case study methodology (Yin, 2014) allows for an in-depth and detailed exploration of the experiences of the research participants, which in this case were the students who participated in the unit using a wiki. These methods allowed for an in-depth investigation of the research themes. Surveys were analysed using Opinio and Mircrosoft Excel to produce descriptive statistics. Thematic analysis of the focus group transcripts data was conducted using the analytical software Dedoose and guided by the three themes flagged for investigation. The authors used a thematic analysis (Braun & Clarke, 2006). The use of Dedoose facilitated the interpretation of the results by keeping a systematic record of the coding, data and source. Braun and Clarke’s (2006) method of thematic analysis identifies, analyses and reports patterns using a process of six steps. The authors used these six steps to analyse the survey and focus group data in order to uncover student perspectives on the three themes under investigation. The six steps include: becoming familiar with the data; generating initial codes; searching for themes; reviewing themes; defining and naming themes, and producing a report. The third author initially coded the transcripts, and the second author recoded approximately 10% to validate the initial coding. The second and third authors achieved high inter-rater reliability of 95% through two moderation sessions. The two authors reached a consensus on the coding and on the inclusion of data. Ethics This project was approved for human research ethics by Swinburne’s Human Research Ethics Committee (SUHREC) and follows the Australian Government’s National Statement on Ethical Conduct in Human Australasian Journal of Educational Technology, 2017, 33(5). 19 Research (2007). The data used in this study was de-identified prior to researchers accessing the primary data to ensure confidential treatment of participants’ data. Results and discussion We report on students’ perspectives on the three themes under investigation: transparency, benchmarking, and feedback. The results are divided accordingly into three sections. These findings are supported by a selection of representative quotes from student surveys and interviews to illustrate each of the major points. Transparency In order to explore how students felt about the transparency afforded by the wiki, students were asked if using the wiki had enabled them to clarify and highlight individual contributions. Results in Figure 1 reveal that most students felt wikis had assisted in clarifying or highlighting individual contributions. Over one third of students (37%) felt the wiki provided good or excellent opportunities to clarify and highlight the contributions of individuals, while 31% felt the wiki had provided some opportunities to do so. Figure 1. Student responses when asked if using a wiki had given them opportunities to be involved in an environment that enabled them to clarify/highlight individual contributions. Results in Figure 2 reveal that students generally felt using a wiki had encouraged their fellow students to be more honest with the work they presented. Twenty-three percent of students strongly felt this to be the case, while almost half of the students (47%) felt the wiki had to some extent encouraged their peers to be more honest with the work they presented. Australasian Journal of Educational Technology, 2017, 33(5). 20 Figure 2. Student responses when asked if using a wiki had encouraged fellow students to be more honest with the work they presented. The results in Figure 3 reveal that students generally felt that the transparency of the wiki assisted group members to get started and do their work. Further, students felt that the wiki allowed them to monitor group members’ participation and progress. One surveyed student stated: “It shows who edited and when”. Figure 3. Student responses when asked about the functionality of wiki to assist with aspects of transparency. Some students felt that the transparent nature of the wiki encouraged group members to do an equal share of the work. By making all contributions transparent to the group throughout the duration of the group work process, students could not conceal their efforts. A surveyed student commented that it is “good to pressure all the members to do an equal amount of work”. Students observed that the transparency afforded by the wiki could act as a deterrent for students wishing to underperform: 0 5 10 15 20 25 30 Making your group members work Encouraging group members to get started Monitoring the participation/progress of other group members Pe rc en ta ge 1 2 3 4 5 1 = least capability 5 = best capability Australasian Journal of Educational Technology, 2017, 33(5). 21 I think it helped because you could check your individual contribution to the wiki, so everyone could check their contribution and if someone hadn’t met that, I think they felt worried. Because for me I knew that you had to contribute 1500 words, so I made sure that at least I had 1500 put into [the wiki]. (focus group) Students generally felt that the transparency of the wikis provided peace of mind that individual contributions were easily identified: It’s the clarity of things that made it really good because you had a clear structure and you had a clear breakdown of who’s responsible for what et cetera. So that really helped ... I think the good thing about wiki is it makes everyone accountable and there needs to be more of that. (focus group) Generally, students agreed that using a wiki had helped improve their control of and confidence in how to manage the group work. Individual contributions were made visible to the entire group which motivated students to contribute equally. A focus group student stated: “If one person falls behind the entire group suffers, so you’ve got to keep up with the workload. So that was a good thing.” Students largely felt that wikis provided a good way to manage people’s workloads. One interviewed student commented that “[you] understand what you have to do, your tasks, individual tasks”. Students knew what was expected of them, as work tasks for the wiki were allocated. A focus group student commented: “More just to make sure everyone did a similar amount because that’s sort of what we allocated to begin with and there wasn’t one person trying to squeeze it all together.” Cooperation not collaboration An important finding was that the wiki did not encourage student collaboration, but rather provided a platform where students could easily delegate tasks and roles within the larger project. Students commented that the wiki enabled groups to delegate tasks which led to students working separately rather than collaborating: So I think it was a lot easier to work in a group, mostly because we didn’t work in a group as much. Each week we set down individual sections for each group to do which was separate wiki pages, and I guess the content was the same. We’d probably do a similar thing as far as content’s concerned: Allocate different subjects for each student in the group. And as far as putting it together, it was probably a bit easier because we didn’t have to worry about things like formatting in a report and problems with combining different reports into one and things like that. (focus group) Some students acknowledged that the wiki contributed to task delegation and student autonomy rather than student collaboration: The wiki format allowed us to work a lot more autonomously and ideally, especially if we did it again, would actually very much create a road map at the beginning and structure it in such a way that we would actually produce our individual pages and then review them as a group as an ongoing process. (focus group) This minimised the time groups needed to spend working together. Some even commented that this task delegation led students to feel as though they were working as individuals rather than as a group: We met every week when we went for the tutorials because we were all in the same tutorial, but we were like, the meeting was just like distributing the task that has to be done. But then there was no time together to actually work on the wiki. It was just like individual work. (focus group) These comments reflect research findings of how wikis are used by students. Wikis facilitate cooperation and do not encourage collaboration per se unless specifically designed for that purpose (Cole, 2009; Elgort et al., 2008; Hadjerrouit, 2012; Judd et al., 2010; Karasavvidis, 2010; Witney & Smallbone, 2011). In this Australasian Journal of Educational Technology, 2017, 33(5). 22 particular project, wikis were not used to directly foster collaboration, but to assist with issues such as task delegation, work quality, and transparency. Benchmarking Using a wiki offered students the opportunity to view what others were doing with their assignments, in terms of content, quality and progress. Students were asked if they felt their fellow students were more honest with the work they presented as a result of this. In responding to the question students revealed some insight into how they felt about wikis as a tool for benchmarking. Despite the student concerns regarding copying, the majority of students, at 81%, felt it was a fair requirement to show their project and group activities to other students using a wiki. See Figure 4. Figure 4. Student responses when asked if it was a fair requirement to show their project and group activities to other students through using a wiki. Students generally felt that looking at other groups’ wikis had assisted them to improve their projects and get started. Figure 5 illustrates that students generally felt the wiki had helped them get started and improve their work. Just a quarter of the surveyed students felt that the wiki had little or no effect on improving their projects or getting them started. Australasian Journal of Educational Technology, 2017, 33(5). 23 Figure 5. Student responses when asked how looking at other groups wikis had assisted with improving the projects and helping them to get started. Student survey comments reflected their feelings that it was fair that others could view their work to monitor if they were on track. One stated “I don't care what really are others doing, similar things I guess” while another commented that it was “fair because I am not jealous if others learn from what I know and they should not be jealous if I learn something from them”. One surveyed student felt that sharing work was an authentic experience, stating: “Your work will always be exposed. It is a good experience.” Another stated that they liked to share their knowledge. The benefits of being able to compare their progress with others, was noted by students. An interviewed student stated: “You could see where everyone else is up to … and where you guys should be up to … so that was good.” Surveyed students commented that: “We started early and felt we were ahead [and] referring to other groups work and comparing with them is very useful”. Some students felt, however, that it could also be negative comparing work and progress with that of others with one surveyed student commenting that the wiki made them “more stressed”. Students generally valued being able to compare their work with that of other groups to get new ideas about topics, content or structure. Surveyed student comments include “[it] helped a little to see what I was meant to do ... topics to discuss”, “[it] helped with the idea, for the structure of the report”, “being able to see the structure of other students’ meant I could keep my project focused”, and “other wikis gave me the idea of how to work”. An interviewed student reported that students got specific ideas from each other’s work “like putting pictures in the introduction … a few were doing that and then after the first feedback everyone was doing that”. One of the most commonly discussed benefits of using a wiki was being able to view the work of others. Students used other groups’ wikis as a means of checking their work standards with those of other groups with some stating that seeing other groups’ work was “the biggest benefit of the wiki”. Another student commented: A pro for it though was that you could easily collaborate with a lot of other people instantaneously … if you were ever unsure on structure or generally, you could always go have a look at somebody else’s and go, “Oh, that’s a really good structure they’ve got set up,” where with physical, with Word documents, you have to go find the person themselves. (focus group) Australasian Journal of Educational Technology, 2017, 33(5). 24 Students generally felt that the wiki had increased their awareness of the quality and the level of work expected. A survey comment reported that the wiki was “good as you can get a general idea what is expected”. Many students furthermore felt the transparency provided by the wiki format had lifted some students’ work efforts and the overall quality of the work: I think it [wiki] forces students to develop maybe a sense of professionalism because maybe other students are up to a certain level so they have to be up to that level to have continuously good content all throughout maybe. I think that was very beneficial. (focus group) Students were able to benchmark the quality of their work against others, thereby increasing the level: Yeah, just kind of like quality [of ideas], like if you think you’ve written enough for an executive summary and you go have a look at someone else’s report, it’s three times the size, you go, “Oh wow, we’re going to get a worse mark because he’s going to read this one and then read ours”. (focus group) One student raised the issue of how the wiki encouraged competitiveness amongst certain students and thereby increased the quality of work: It’s very competitive because you see how other groups are doing, then you’re trying to improve to beat them so the overall quality increases I think as well, that’s important. (focus group) A further comment revealed that the wiki task had helped to improve the quality of some students’ written language and expression: One thing I’ve noticed with you [name] when we first initiated the whole thing, some people’s linguistic and writing skills weren’t up to the level of some of the other students and I think that forced the students who weren’t up to par to improve. I’m not trying to embarrass you but maybe some writing needs to be a bit more formal. (focus group) Some students, however, felt that they were unable to identify whether others’ work was of a low or high quality with survey comments including “so many groups and you don't know which one is good” and it was “the blind leading the blind”. This finding reflects research that shows that students – particularly those at the lower end of the performance scale – tend to have trouble identifying when their own work is of a good quality (Boud, Lawson, & Thompson, 2015). This issue of not recognising the expected standards may be resolved by providing students with exemplars of good practice or incorporating more detailed tutor feedback early on in the feedback process. The main negatives cited by students related to copying. Students raised concerns about the wiki providing opportunities for copying with survey comments including: “it is possible for other less organised groups to copy the work of other groups” and “you can read someone else work and re-word”. A further surveyed participant stated that it is not only content that can be copied but styles and links to research. Students especially felt that copying could be facilitated by those who uploaded their work to the wiki close to the deadline. A surveyed student commented that those “who left the work to the last minute can copy other people’s ideas, especially when they are doing the same company”. An interviewed student stated that groups could work in word and upload their contribution to the wiki in the last week in order to avoid it being copied by others: “People have it on Word and not on wiki then upload it in the last week.” Some students felt there was not enough work available to benchmark against as some groups did not upload their work to the wiki until the last minute. One stated: “Many groups left their wiki till the last day before it was assessed so there wasn't much of an example to look at.” This practice of leaving work until the last minute is a practice commonly found in the research on wiki use in higher education (Forte & Bruckman, 2006; Hadjerrouit, 2012; Judd et al., 2010; Sampaio‐Maia et al., 2014). Solutions may include clearly stating deadlines for wiki contributions and making contributions an assessed requirement. Australasian Journal of Educational Technology, 2017, 33(5). 25 Not all students, however, held the concern that transparency could lead to copying. There were those who recognised that should copying occur, it would be apparent to all members of the wiki that this had happened. Others recognised that it may be easier to identify copying through the use of wikis. Survey comments included: “any copying of ideas can be immediately apparent to members on the wiki pages” and “[wikis make it] easier to see if students are copying and pasting other people work”. Feedback In order to investigate how students felt about the feedback component of the wiki, they were asked whether the wiki had provided them with opportunities to be involved with peer assessment and provide feedback. Results in Figure 6 reveal that students felt they had been given opportunities to partake in peer assessment and feedback practices with almost two thirds of responding that they had either good or excellent opportunities to do so. Over a quarter of students felt they had some opportunities and just 10% felt the wiki provided little or no opportunities for peer assessment and feedback opportunities. Figure 6. Student responses when asked if using a wiki had given them opportunities to be involved in peer assessment and provide feedback. When asked about what it meant to view the work of others, one student reported that in response to viewing others’ work, they had taken action that was beneficial to their own work stating: “You need to respond to feedback which involves you more in your own work.” Students were asked to evaluate how the wiki performed when used for a number of specific roles. Results are shown in Figure 7. Over two thirds of the students agreed or strongly agreed that the wiki was useful for giving and receiving feedback. Just over half the students reported that the wiki made the feedback more effective, thereby enabling them to improve their work. Agreement declined to just over 40% as to the effectiveness of the wiki as an environment for supporting frequent feedback. Almost a third of all students agreed that the wiki had raised the quality of feedback compared with anonymous discussion. This finding indicates that the wiki as a tool in itself does not facilitate improvements in the quality or frequency of comments, much in line with the literature (Hadjerrouit, 2012; Judd et al., 2010; Karasavvidis, 2010). As such, quality and frequent feedback needs to be incorporated in activities in the unit through the learning design of the unit. Furthermore, providing quality feedback is a skill that can and should be developed in students (Falchikov, 2013; Merry, Price, Carless, & Taras, 2013; Molloy & Boud, 2014; Nicol, 2011; Nicol, Thomson, & Breslin, 2014). Australasian Journal of Educational Technology, 2017, 33(5). 26 Figure 7. Student responses when asked to evaluate the wiki when used for specific roles. Students were asked which types of feedback had been most helpful to them. Results are reported in Figure 8. Over 70% of students found the teaching team’s feedback to be most helpful, while almost a third of students found the feedback from other groups helpful. Figure 8. Student responses when asked which types of feedback had helped them improve their projects. Students were asked to indicate which feedback iteration had been most useful to them: the first, second, or final iterations. Results are reported in Figure 9. While students had already indicated that feedback had been valuable to them (see Figure 8), a minority of students in each case indicated that different types of feedback had been more useful than others. In all cases, students had found staff feedback more helpful than feedback from their peers. This reflects the findings in Figure 7 and Figure 8, showing a need to develop the quality of feedback that students are capable of providing to each other. On average, 23% of students found the student feedback most helpful across the three feedback iterations, with the value of feedback declining slightly across the three iterations of feedback throughout the semester. Student responses regarding staff feedback iterations were similar. On average, 33% of students found them useful. 0 5 10 15 20 25 30 35 40 45 50 Useful for giving and receiving feedback Raised the quality of feedback compared with anonymous feedback Made feedback more effective and work has improved as a result Provides an environment for frequent feedback Pe r ce nt Strongly disgree Disagree Neither agree or disagree Agree Strongly agree 72% 32% 29% 0 10 20 30 40 50 60 70 80 Teaching team's feedback Other groups' feedback Your team members' feedback Pe r c en t Australasian Journal of Educational Technology, 2017, 33(5). 27 Figure 9. Student responses when asked to indicate which items of feedback had been useful to them. The majority of the interviewed students claimed to use the feedback provided to them, and students largely agreed that the feedback helped their work. Students felt that regular feedback through the wiki had provided a more supportive teaching environment than a standard report submission. Survey comments include “having feedback from tutor as project progressed was helpful”, while others requested “more regular feedback” revealing the importance that students place on feedback. Students often expressed dissatisfaction with the quality of feedback they received from their peers. Generally, the interviewed students were dissatisfied with the feedback they received from their peers stating that “people didn’t really take time and consideration”. Students felt that peer comments were “very short and vague” presumably due to the perception that “there was no weight to it [providing feedback]”. Students were also dissatisfied with the feedback provided by tutors. One felt that the tutor feedback they received was inadequate stating “ours [feedback] was just like a smiley face from a tutor and you look at it and you think ‘How’s this going to help us?’” Students also generally felt that feedback was often provided too late for them to use to effectively improve their work. Interviewed students stated that feedback was “done in the last minute” and furthermore, that “everyone leaves it to the last minute”. In addition to providing feedback at the last minute, students also uploaded their work at the last minute. This practice made it difficult for students to receive feedback on the most up-to-date work: We had feedback that we provided other groups and they provided us, and then our work component was due for assessment the following week, and that was both good and bad because it was good, I guess, theoretically because you get the feedback beforehand, but like I was saying, we really didn’t upload our work until after the feedback had already been given, so we were getting feedback that wasn’t relevant because we’d already done most of the things that were being said. (focus group) Despite wanting to receive quality and timely feedback on their own work, students admitted that they themselves did not make the effort to submit quality or timely feedback to their peers. An interviewed student admitted: “I was doing the feedback. I did it on the last day of the deadline. It was very bad feedback, I just said basic stuff.” When asked how the feedback could be improved, students wanted more in-depth and detailed feedback and they wanted it earlier in the course of the assessment. In order to do this, students suggested integrating feedback formally into the assessment – though this is already a unit requirement. Student focus group comments included: “Make the feedback carry marks and people would put more time and consideration 26% 34% 24% 32% 20% 33% 0 5 10 15 20 25 30 35 1st feedback from a student 1st feedback from staff 2nd feedback from a student 2nd feedback from staff Final feedback from a student Final feedback from staff Pe r ce nt Australasian Journal of Educational Technology, 2017, 33(5). 28 into what they write.” The discrepancy between what students want from a feedback process and what they are willing to provide is an interesting area for future research. Developing students’ abilities to provide quality feedback is a step towards achieving improved feedback and ultimately improved work (Falchikov, 2013; Merry et al., 2013; Molloy & Boud, 2014; Nicol, 2011; Nicol et al., 2014). Conclusion and future work This research explored how students perceived that the use of a wiki for a group assignment impacted the group work process in terms of the transparency of work contributions, the benchmarking of work quality and the provision of feedback on peers’ work. Students reported that the transparency afforded by the wiki allowed for the identification of individual contributions and assisted in managing roles and workloads among group members. Results indicate that the majority of students appreciated the transparency of the wiki in having potential benefits for their learning experience and some viewed the transparency as a professional expectation. In terms of the group work, students reported that the transparency of the tool clarified group members’ tasks, workloads and individual contributions. This benefited the management of group work responsibilities. The transparency created benchmarking opportunities through which students could get ideas, pace their progress and increase the quality of their work by viewing other students’ work. Additionally, benchmarking enabled students to use their judgment skills and develop their own standards. A common concern cited by students related to copying, however, other students noted that the transparency of the wiki itself acted as a deterrent for copying. Our findings indicate, however, that the wiki in itself did not facilitate student collaboration. Instead the wiki was used to allow students to cooperate and delegate tasks which they then worked on in relative isolation. Although collaboration was not an intended goal of the wiki assessment, we conclude that in order to encourage collaboration using a wiki, solid learning design is needed to scaffold and guide students to work constructively and efficiently together in a wiki group task. Using a wiki was a powerful tool to encourage ongoing feedback from staff and students. It appears that the majority of students found detailed feedback valuable and used the provided feedback to guide their work. A common complaint, however, was that the quality of feedback was regularly inadequate – particularly feedback provided by students – and that it was often delivered too late to be of use in improving students’ work. Students themselves admitted to not delivering quality or timely feedback. These findings suggest that students may benefit from guidance surrounding how to provide peers with quality feedback which can lead to improvements. Learning design can further be used to ensure that feedback is provided throughout the semester and in a timely way that allows students to act on it. The introduction of explicit feedback requirements and deadlines may assist in this. In this project, the use of a wiki appears to be an effective method for increasing communication with and amongst students continually throughout the semester, and for increasing the quality of submissions. However, new challenges need to be addressed. In particular, collaboration and group work needs to be better understood and explored, as it seems meaningful collaboration among students is less common than what educators often expect. Students also need to be trained in how to deliver effective feedback to peers to improve the quality of their feedback and subsequently students’ work. Implementing wikis for peer assessment may assist educators to facilitate quality and timely feedback to students – particularly those under time constraints and facing the challenges of assessing large student cohorts. This project is in its second iteration and the project leader plans to continue refining the use of wikis in future units. In particular, future iterations aim to improve the quality and timing of feedback as well as increase student collaboration through robust learning design. References Allwardt, D. E. (2011). Writing with wikis: A cautionary tale of technology in the classroom. Journal of Social Work Education, 47(3), 597-605. Baltzersen, R. K. (2010). Radical transparency: Open access as a key concept in wiki pedagogy. Australasian Journal of Educational Technology, 26(6), 791-809. https://doi.org/10.14742/ajet.1043 https://doi.org/10.14742/ajet.1043 Australasian Journal of Educational Technology, 2017, 33(5). 29 Bazeley, P. (2012). Integrative analysis strategies for mixed data sources. American Behavioral Scientist, 56(6), 814-828. https://doi.org/10.1177/0002764211426330 Blumenfeld, P. C., Marx, R. W., Soloway, E., & Krajcik, J. (1996). Learning with peers: From small group cooperation to collaborative communities. Educational Researcher, 25(8), 37-39. https://doi.org/10.2307/1176492 Boud, D., Lawson, R., & Thompson, D. G. (2015). The calibration of student judgement through self- assessment: disruptive effects of assessment patterns. Higher Education Research & Development, 34(1), 45-59. https://doi.org/10.1080/07294360.2014.934328 Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa Caple, H., & Bogle, M. (2013). Making group assessment transparent: what wikis can contribute to collaborative projects. Assessment & Evaluation in Higher Education, 38(2), 198-210. https://doi.org/10.1080/02602938.2011.618879 Carr, T., Morrison, A., Cox, G., & Deacon, A. (2007). Weathering wikis: Net-based learning meets political science in a South African university. Computers and Composition, 24(3), 266-284. https://doi.org/10.1016/j.compcom.2007.06.001 Clougherty, R., & Wells, M. (2008). Use of wikis in chemistry instruction for problem-based learning assignments: An example in instrumental analysis. Journal of Chemical Education, 85(10), 1446- 1448. https://doi.org/10.1021/ed085p1446 Cole, M. (2009). Using wiki technology to support student engagement: Lessons from the trenches. Computers & Education, 52(1), 141–146. https://doi.org/10.1016/j.compedu.2008.07.003 Cunningham, C., O'Donoghue, G., & Jennings, D. (2016). Introduction of wikis to foster collaboration in health professional education. Focus on Health Professional Education: A Multi-disciplinary Journal, 17(1), 30-44. https://doi.org/10.11157/fohpe.v17i1.120 Dalke, A. F., Cassidy, K., Grobstein, P., & Blank, D. (2007). Emergent pedagogy: Learning to enjoy the uncontrollable - and make it productive. Journal of Educational Change, 8(2), 111-130. https://doi.org/10.1007/s10833-007-9021-2 Dalsgaard, C., & Paulsen, M. F. (2009). Transparency in cooperative online education. International Review of Research in Open and Distance Learning, 10(3). Retrieved from https://www.irrodl.org/index.php/irrodl/article/view/671/1267 De Wever, B., Hämäläinen, R., Voet, M., & Gielen, M. (2015). A wiki task for first-year university students: The effect of scripting students' collaboration. The Internet and Higher Education, 25, 37- 44. https://doi.org/10.1016/j.iheduc.2014.12.002 De Wever, B., Van Keer, H., Schellens, T., & Valcke, M. (2011). Assessing collaboration in a wiki: The reliability of university students’ peer assessment. Internet and Higher Education, 14(4), 201-206. https://doi.org/10.1016/j.iheduc.2011.07.003 Ebner, M., Kickmeier-Rust, M., & Holzinger, A. (2008). Utilizing wiki-systems in higher education classes: A chance for universal access? Universal Access in the Information Society, 7(4), 199-207. https://doi.org/10.1007/s10209-008-0115-2 Elgort, I., Smith, A. G., & Toland, J. (2008). Is wiki an effective platform for group course work? Australasian Journal of Educational Technology, 24(2), 195-210. https://doi.org/10.14742/ajet.1222 Falchikov, N. (2013). Improving assessment through student involvement: Practical solutions for aiding learning in higher and further education. London, UK: Routledge Falmer. Foley, B., & Chang, T. (2008, March). Wiki as a professional development tool. Paper presented at the Society for Information Technology & Teacher Education International Conference, Las Vegas, NV. Retrieved from https://www.csun.edu/~bfoley/AERA_Wiki.pdf Forte, A., & Bruckman, A. (2006, June - July). From Wikipedia to the classroom: Exploring online publication and learning. Paper presented at the 7th International Conference on Learning Sciences, Indiana University, Bloomington, IN. Retrieved from https://dl.acm.org/citation.cfm?id=1150061 Grant, L. (2009). I DON’T CARE DO UR OWN PAGE!’ A case study of using wikis for collaborative work in a UK secondary school. Learning, Media and Technology, 34(2), 105-117. https://doi.org/10.1080/17439880902923564 Hadjerrouit, S. (2012, October). Using wikis to foster collaborative writing: Exploring influencing factors to successful implementation. Paper presented at the IADIS International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2012), Madrid, Spain. Retrieved from https://www.learntechlib.org/p/132366 https://doi.org/10.1177/0002764211426330 https://doi.org/10.2307/1176492 https://doi.org/10.1080/07294360.2014.934328 https://doi.org/10.1191/1478088706qp063oa https://doi.org/10.1080/02602938.2011.618879 https://doi.org/10.1016/j.compcom.2007.06.001 https://doi.org/10.1021/ed085p1446 https://doi.org/10.1016/j.compedu.2008.07.003 https://doi.org/10.11157/fohpe.v17i1.120 https://doi.org/10.1007/s10833-007-9021-2 https://www.irrodl.org/index.php/irrodl/article/view/671/1267 https://doi.org/10.1016/j.iheduc.2014.12.002 https://doi.org/10.1016/j.iheduc.2011.07.003 https://doi.org/10.1007/s10209-008-0115-2 https://doi.org/10.14742/ajet.1222 https://www.csun.edu/%7Ebfoley/AERA_Wiki.pdf https://dl.acm.org/citation.cfm?id=1150061 https://doi.org/10.1080/17439880902923564 https://www.learntechlib.org/p/132366 Australasian Journal of Educational Technology, 2017, 33(5). 30 Ioannou, A., & Artino, A. R. (2009). Wiki and threaded discussion for online collaborative activities: Students' perceptions and use. Journal of Emerging Technologies in Web Intelligence, 1(1), 97-105. Retrieved from https://www.jetwi.us/uploadfile/2014/1230/20141230113630354.pdf Judd, T., Kennedy, G., & Cropper, S. (2010). Using wikis for collaborative learning: Assessing collaboration through contribution. Australasian Journal of Educational Technology, 26(3), 341-354. https://doi.org/10.14742/ajet.1079 Kalfa, S., & Taksa, L. (2015). Cultural capital in business higher education: reconsidering the graduate attributes movement and the focus on employability. Studies in Higher Education, 40(4), 580-595. https://doi.org/10.1080/03075079.2013.842210 Karasavvidis, I. (2010). Wiki uses in higher education: Exploring barriers to successful implementation. Interactive Learning Environments, 18(3), 219-231. https://doi.org/10.1080/10494820.2010.500514 Kear, K., Woodthorpe, J., Robertson, S., & Hutchison, M. (2010). From forums to wikis: Perspectives on tools for collaboration. The Internet and Higher Education, 13(4), 218-225. https://doi.org/10.1016/j.iheduc.2010.05.004 Larusson, J. A., & Alterman, R. (2009). Wikis to support the “collaborative” part of collaborative learning. International Journal of Computer-Supported Collaborative Learning, 4(4), 371-402. https://doi.org/10.1007/s11412-009-9076-6 Lin, H., & Kelsey, K. D. (2009). Building a networked environment in wikis: The evolving phases of collaborative learning in a wikibook project. Journal of Educational Computing Research, 40(2), 145- 169. https://doi.org/10.2190/EC.40.2.s Lund, A. (2008). Wikis: A collective approach to language production. ReCALL, 20(1), 35-54. https://doi.org/10.1017/S0958344008000414 Merry, S., Price, M., Carless, D., & Taras, M. (2013). Reconceptualising feedback in higher education: Developing dialogue with students. London: Routlege. Molloy, E. K., & Boud, D. (2014). Feedback models for learning, teaching and performance Handbook of research on educational communications and technology (pp. 413-424). New York, NY: Springer. Nagarajan, S., & Edwards, J. (2014). Is the graduates attribute approach sufficient to develop work ready graduates? Journal of Teaching and Learning for Graduate Employability, 5(1), 12-28. https://doi.org/10.21153/jtlge2014vol5no1art56s Nejkovic, V., & Tosic, M. (2014). Wiki learning system patterns for academic courses. Computer Applications in Engineering Education, 22(4), 678-685. https://doi.org/10.1002/cae.21559 Neumann, D. L., & Hood, M. (2009). The effects of using a wiki on student engagement and learning of report writing skills in a university statistics course. Australasian Journal of Educational Technology, 25(3), 382-398. https://doi.org/10.14742/ajet.1141 Nicol, D. (2011). Developing the students’ ability to construct feedback. Gloucester: Quality Assurance Agency for Higher Education. Retrieved from https://www.enhancementthemes.ac.uk/docs/publications/developing-students-ability-to-construct- feedback.pdf Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: A peer review perspective. Assessment & Evaluation in Higher Education, 39(1), 102-122. https://doi.org/10.1080/02602938.2013.795518 O'Connor, E. (2010, March). The use of a wiki in teacher education: How does learning and instruction change when work can go public? Paper presented at the Society for Information Technology & Teacher Education International Conference, Chesapeake, VA. Retrieved from https://www.learntechlib.org/p/33797 Osman-Schlegel, L., Fluker, G., & Cheng, S. T. (2011, December). Working collaboratively in a group assignment using a Mediawiki for an architecture and construction management undergraduate unit. Paper presented at the Changing Demands, Changing Directions (ASCILITE2011), Hobart, Australia. Retrieved from https://www.ascilite.org/conferences/hobart11/downloads/ProceedingsV3.pdf Popescu, E. (2014, February). Using wikis to support project-based learning: A case study. Paper presented at the Fourteenth IEEE International Conference on Advanced Learning Technologies (ICALT2014), Athens, Greece. https://doi.org/10.1109/ICALT.2014.94 Putro, I. H., Carbone, A., & Sheard, J. (2014, January). Developing a framework to assess students' contributions during wiki construction. Paper presented at the The Sixteenth Australasian Computing Education Conference (ACE2014), Auckland, New Zealand. Retrieved from https://crpit.com/confpapers/CRPITV148Putro.pdf https://www.jetwi.us/uploadfile/2014/1230/20141230113630354.pdf https://doi.org/10.14742/ajet.1079 https://doi.org/10.1080/03075079.2013.842210 https://doi.org/10.1080/10494820.2010.500514 https://doi.org/10.1016/j.iheduc.2010.05.004 https://doi.org/10.1007/s11412-009-9076-6 https://doi.org/10.2190/EC.40.2.s https://doi.org/10.1017/S0958344008000414 https://doi.org/10.21153/jtlge2014vol5no1art56s https://doi.org/10.1002/cae.21559 https://doi.org/10.14742/ajet.1141 https://doi.org/10.1080/02602938.2013.795518 https://www.learntechlib.org/p/33797 https://www.ascilite.org/conferences/hobart11/downloads/ProceedingsV3.pdf https://doi.org/10.1109/ICALT.2014.94 https://crpit.com/confpapers/CRPITV148Putro.pdf Australasian Journal of Educational Technology, 2017, 33(5). 31 Robertson, I. (2008). Learners' attitudes to wiki technology in problem based, blended learning for vocational teacher education. Australasian Journal of Educational Technology, 24(4), 425-441. https://doi.org/10.14742/ajet.1202 Rosenshine, B. V., & Meister, C. (1992). The use of scaffolds for teaching less structured cognitive tasks. Educational Leadership, 49(7), 26-33. Retrieved from https://www.ascd.org/ASCD/pdf/journals/ed_lead/el_199204_rosenshine.pdf Sampaio‐Maia, B., Maia, J. S., Leitão, S., Amaral, M., & Vieira‐Marques, P. (2014). Wiki as a tool for Microbiology teaching, learning and assessment. European Journal of Dental Education, 18(2), 91- 97. https://doi.org/10.1111/eje.12061 Thomas, I., & Day, T. (2014). Sustainability capabilities, graduate capabilities, and Australian universities. International Journal of Sustainability in Higher Education, 15(2), 208-227. https://doi.org/10.1108/IJSHE-05-2012-0046 van Merriënboer, J. J. G., Kirschner, P. A., & Kester, L. (2003). Taking the load off a learner’s mind: Instructional design for complex learning. Educational Psychologist, 38(1), 5-13. https://doi.org/10.1207/S15326985EP3801_2 Witney, D., & Smallbone, T. (2011). Wiki work: Can using wikis enhance student collaboration for group assignment tasks? Innovations in Education and Teaching International, 48(1), 101-110. https://doi.org/10.1080/14703297.2010.543765 Yin, R. K. (2014). Case study research: Design and methods. Thousand Oaks CA: Sage. Zheng, B., Niiya, M., & Warschauer, M. (2015). Wikis and collaborative learning in higher education. Technology, Pedagogy and Education, 24(3). https://doi.org/10.1080/1475939X.2014.948041 Zorko, V. (2009). Factors affecting the way students collaborate in a wiki for English language learning. Australasian Journal of Educational Technology, 25(5), 645–665. https://doi.org/10.14742/ajet.1113 Zorn, S., Sasaki, R., Qian, D., & Chase, A. M. (2016, August). Group work in online business education – pain or gain? Paper presented at the Hawaii International Conference on Education, Honolulu, HI. Retrieved from https://www.hiceducation.org/ Corresponding author: Bella Ross, bella.ross@monash.edu Australasian Journal of Educational Technology © 2017. Please cite as: Abdekhodaee, A., Chase, A.-M. & Ross, B. (2017). Wikis for group work: Encouraging transparency, benchmarking, and feedback. Australasian Journal of Educational Technology, 33(5), 15- 31. https://doi.org/10.14742/ajet.2829 https://doi.org/10.14742/ajet.1202 https://www.ascd.org/ASCD/pdf/journals/ed_lead/el_199204_rosenshine.pdf https://doi.org/10.1111/eje.12061 https://doi.org/10.1108/IJSHE-05-2012-0046 https://doi.org/10.1207/S15326985EP3801_2 https://doi.org/10.1080/14703297.2010.543765 https://doi.org/10.1080/1475939X.2014.948041 https://doi.org/10.14742/ajet.1113 http://www.hiceducation.org/ mailto:bella.ross@monash.edu https://doi.org/10.14742/ajet.2829 Wikis for group work: Encouraging transparency, benchmarking, and feedback Introduction A wiki was introduced to a unit, based on previous observations by the first author that group work often resulted in group conflicts, and poorly structured and overlapping work of varying quality. The aim of this research was to investigate student p... Literature review: Using wikis in higher education Methodology Participants and context Data collection and analysis Ethics Results and discussion Transparency Cooperation not collaboration Benchmarking Feedback Conclusion and future work References