editorial27-5.pdf Australasian Journal of Educational Technology Volume 27, Number 5 Special issue, 2011 ISSN 1449-5554 (online) Contents Editorial 27(5): Preface to the Special issue - Assessing students' Web 2.0 activities in higher education ........................................................................ iii-ix Jenny Waycott and Judy Sheard Marking machinima: A case study in assessing student use of a Web 2.0 technology ..................................................................................... 765-780 Graham Barwell, Chris Moore and Ruth Walker Design-grounded assessment: A framework and a case study of Web 2.0 practices in higher education ...................................................................... 781-797 Yu-Hui Ching and Yu-Chang Hsu Emphasising assessment ‘as’ learning by assessing wiki writing assignments collaboratively and publicly online .......................................... 798-812 Anne Davies, Kerry Pantzopoulos and Kathleen Gray Finding the discipline: Assessing student activity in Second Life .................. 813-828 Scott Grant and Rosemary Clerehan Can Web 2.0 technology assist college students in learning English writing? Integrating Facebook and peer assessment with blended learning ................. 829-845 Ru-Chu Shih Using Web 2.0 to teach Web 2.0: A case study in aligning teaching, learning and assessment with professional practice ..................................... 846-862 John Terrell, Joan Richardson and Margaret Hamilton The Australasian Journal of Educational Technology (AJET) is a refereed research journal published 6 times per year by the Australasian Society for Computers in Learning in Tertiary Education (ascilite). AJET retired its printed version (ISSN 1449-3098) at the end of Volume 23, 2007, and from Volume 24, 2008, the journal is open access, online only (ISSN 1449-5554), and does not have paid subscriptions. © 2011 Authors retain copyright in their individual articles, whilst copyright in AJET as a compilation is retained by the publisher. Except for authors reproducing their own articles, no part of this journal may be reprinted or reproduced without permission. For further details, and for details on submission of manuscripts and open access to all issues of AJET published since the journal's foundation in 1985, please see http://www.ascilite.org.au/ajet/ i i Australasian Journal of Educational Technology, 2011, 27(Special issue, 5) For editorial inquiries, contact the Editor, Associate Professor Catherine McLoughlin, School of Education (ACT), Australian Catholic University, PO Box 256, Dickson ACT 2602, Australia. Email: Catherine.McLoughlin@acu.edu.au, Tel: +61 2 6209 1100 Fax +61 2 6209 1185. For review process, production, website and business matters, contact the Production Editor, Dr Roger Atkinson, 5/202 Coode Street, Como WA 6152, Australia. Email: rjatkinson@bigpond.com, Tel: +61 8 9367 1133. Desktop publishing (PDF versions) and HTML by Roger Atkinson. AJET is managed by a Committee comprising ASCILITE Executive nominees, the convenors or nominees from previous ascilite Conferences, and AJET's previous editors and current senior editorial staff. The 2010 Management Committee members are: Professor Mike Keppell, Charles Sturt University, ASCILITE President Dr Philippa Gerbic, Auckland University of Technology, ASCILITE Executive Professor Geoffrey Crisp, University of Adelaide, ASCILITE 2003 Convenor Dr Rob Phillips, Murdoch University, ASCILITE 2004 Convenor Professor Peter Goodyear, University of Sydney, ASCILITE 2006 Convenor Dr Dale Holt, Deakin University, ASCILITE 2008 Convenor Professor Ron Oliver, Edith Cowan University, AJET Editor 1997-2001 Assoc Prof Catherine McLoughlin (Editor), Australian Catholic University Dr Roger Atkinson (Production Editor) AJET's Editorial Board (see http://www.ascilite.org.au/ajet/about/editorial- board.html) reflects the journal's commitment to academic excellence in educational technology and related areas of research and professional practice, our vision of an international journal with an Australasian regional emphasis, and our origins as a professional and learned society publication. Australasian Society for Computers in Learning in Tertiary Education http://www.ascilite.org.au/ Australasian Journal of Educational Technology, 2011, 27(Special issue, 5) iii Editorial 27(5): Preface to the Special issue Jenny Waycott The University of Melbourne Judy Sheard Monash University Editors, Special issue - Assessing students' Web 2.0 activities in higher education In this special issue of the Australasian Journal of Educational Technology we bring together six papers that describe and evaluate assessment tasks that university students, in different disciplinary contexts, have undertaken using Web 2.0 technologies. “Web 2.0” is, of course, an umbrella term that covers a range of technologies and tools, also sometimes referred to as the “social web”. Web 2.0 technologies are linked by features that emphasise open publishing, collaboration, and user-created content. Through technologies such as blogs, wikis, and social networking sites, users can easily share information, collaborate on both large and small scale projects, and review, critique and comment on each other’s contributions. These features mean that social web technologies are frequently lauded as tools t h a t support social constructivist approaches to learning and offer numerous pedagogical advantages for university students, fostering the development of generic skills such as collaboration, writing, and critical analysis (see for example Franklin & van Harmelen, 2007; McLoughlin & Lee, 2007; Melville et al, 2009). However, the same features that make Web 2.0 technologies attractive as learning tools also impose challenges for educators who wish to assess students’ Web 2.0 activities. Until recently, the issue of what constitutes good practice in t h e assessment of students’ Web 2.0 activities in higher education has been largely overlooked (see Gray et al, 2010 for a review). Recent papers that have focused explicitly on Web 2.0 and assessment include Elliott (2008) and Whitelock (2010). Web 2.0 activities can differ substantially from the assessment tasks that university students and teachers are accustomed to, raising both familiar and novel challenges for assessment and academic integrity. How should educators, for instance, assess t h e informal or reflective writing that students create for an unknown audience on t h e web? And how can educators successfully manage large scale collaborative activities, such as when all the students in a class use a collaborative wiki to construct a new textbook (e.g., Baltzersen, 2010)? Web 2.0 activities may also involve students creating or modifying different types of media, such as images and videos. How do educators assess academic standards when students are creating and using non- academic texts and media? Hobart, 4-7 December 2011. http://www.ascilite.org.au/conferences/hobart11/ iv Australasian Journal of Educational Technology, 2011, 27(Special issue, 5) The aim of this special issue is to contribute to the ongoing dialogue about these issues and to highlight the opportunities and challenges associated with assessing students’ social web activities in higher education. In November 2010 we released a call for expressions of interest for members of the Australasian and international higher education community to contribute to this special issue. The call for extended abstracts received a strong response, suggesting that there is a growing interest in t h i s issue, both locally and internationally. In total, 24 abstracts were submitted; the authors of 13 of these abstracts were invited to submit full papers. Following a rigorous peer review process the six papers that follow were selected for publication, from ten full papers submitted (the authors of three accepted abstracts elected to not proceed). These papers demonstrate the diversity of Web 2.0 in higher education, describing a range of assessment tasks that use different Web 2.0 tools, in diverse settings. In each of the papers the authors have addressed specific assessment issues that have been highlighted by their research into the use of Web 2.0 technologies for student assessment tasks in higher education. These issues range from incorporating peer formative assessment into the learning task, to the challenges of assessing set collaborative tasks, and from creating assessment tasks that successfully align w i t h learning objectives and graduate outcomes, to designing assessment that focuses on t h e process, rather than product, of learning. Common to all the papers is an emphasis on collaboration or sharing and peer review, which, as noted above, aligns with the key pedagogical affordances of Web 2.0 for higher education. In some of the papers collaboration is explicit in the design of the assessment task. In others, collaboration is less formal or structured, involving varying degrees of peer review and knowledge sharing. In the first paper, by Graham Barwell, Chris Moore and Ruth Walker, collaboration is a central feature of the learning task, which centres on a technology that is not often exploited in higher education – the World of Warcraft online game. Barwell and colleagues describe an assessment task that makes explicit use of the features and philosophies inherent in the concept of the “social web”. The activity involved cross-disciplinary collaboration: students who were enrolled in English and Digital Media subjects worked together in small groups to create animations (or “machinima”) in World of Warcraft, based on Chaucer tales. The task was unstructured and open ended: students were given free rein to choose their texts, manage the collaborative process, and create the scripts and machinima scenes. The emphasis, in both the task design and assessment, was on understanding and facilitating the process of learning rather than assessing students’ final machinima production. This was an adventurous and novel assessment task. The authors make i t clear that this was a pilot project and was therefore optional for students, with no expectations about students’ ability to complete the final product. The task was designed as much as possible to be learner-centred and the authors, drawing on Wenger’s communities of practice model, demonstrate in this paper that t h e interdisciplinary collaboration was the focus of the assessment. Their evaluation, based on both student and teacher reflections, suggest that this interdisciplinary collaboration is both challenging and rewarding, and the students appear to h a v e valued the learning experience of modifying Chaucer texts to create machinima. Australasian Journal of Educational Technology, 2011, 27(Special issue, 5) v Association for Learning Technology: ALT-C 2011 Thriving in a colder and more challenging climate University of Leeds, UK, 6-8 September 2011 http://www.alt.ac.uk/altc2011/ The assessment task described by Yu-Hui Ching and Yu-Chang Hsu is a more structured, but still collaborative, online activity. Again, this paper focuses on a Web 2.0 tool that is quite different from the blogs and wikis that are normally associated with Web 2.0 in higher education. Ching and Hsu describe a collaborative concept mapping activity in which students used the tool Webspiration to create a map illustrating the processes involved in instructional design. This activity was undertaken as part of an online instructional design course, with students who were geographically dispersed collaborating in small groups. In this paper, Ching and Hsu present a framework that draws on three related theories – sociocultural theory, distributed cognition, and situated cognition – which they use to guide the design and assessment of the assignment. Through this framework, they emphasise the need for student groups to have shared goals when working together on an assessment task. Their study highlights the challenges involved in ensuring collaboration works effectively in an online course, when students are working together at a distance. In their evaluation of the activity, Ching and Hsu show that the collaborative process is instrumental in influencing the outcome of the assessment: the better concept maps were created by students who appeared to work well together. The authors suggest that a focus on both individual learning and shared goals may be necessary, although this may raise workload issues for instructors. They also suggest that Web 2.0 activities in higher education open up the possibility for assessment to focus on both process and product. Like Barwell and colleagues, they argue that it i s important to place emphasis on the social interaction and meaning negotiation t h a t takes place during collaborative learning. However, Ching and Hsu’s assessment differs somewhat in that they also assessed the final product that students created during this learning activity, the concept maps. Anne Davies, Kerry Pantzopoulos and Kathleen Gray present a different, but equally novel, case study in which students used a shared wiki to present their work. W h a t is particularly interesting about this case study is that the students were tertiary educators, enrolled in a subject that focused on assessment as part of a Graduate Certificate in Tertiary Education. That is, in this paper Davies and colleagues examine an assessment task in which students were learning about assessment. During the task, students engaged in reflective practice: they used the shared wiki to “sketch, thread and theorise” about their own experiences of conducting assessment tasks with tertiary students. While this task did not involve the same type of collaborative group work that is discussed in the first two papers, it did involve 5th Asia Pacific Conference on Educational Integrity The University of Western Australia Perth, 26-28 September 2011 http://www.catl.uwa.edu.au/5apcei vi Australasian Journal of Educational Technology, 2011, 27(Special issue, 5) Nusa Dua, Bali, Indonesia, 2-5 October 2011. http://www.ut.ac.id/icde2011/ knowledge-sharing and peer review. Students were expected to review and comment on their peers’ work, a practice that was made possible by using a shared wiki for the task. By fostering this process of peer review, the authors aimed to establish a collaborative learning community of reflective practitioners. Another notable feature of this task is that it was a high-stakes assignment, worth 100% of t h e students’ marks in the subject. It was therefore important that this task fostered students’ learning. The authors argue that the task was designed to promote assessment ‘as’ learning, that is, assessment that facilitates students’ awareness of their own learning processes and strategies. This was achieved through t h e emphasis on reflective learning and knowledge sharing in this wiki-writing assignment. Scott Grant and Rosemary Clerehan describe what is perhaps the most structured of the six learning activities in this special issue. In this assignment, first-year Chinese language students participated in two laboratory classes during which t h e y completed learning activities in the virtual world Second Life. While there is some debate about whether virtual worlds and online games such as World of W a r c r a f t belong in the category of “Web 2.0”, we include them here because they share w i t h typical Web 2.0 technologies an emphasis on user collaboration and content creation. In the activity described by Grant and Clerehan, collaboration is again a central feature of the task. Students worked in small groups to complete a number of tasks, in which they needed to communicate with other players in the virtual world using Chinese characters and the vocabulary they had been learning in class. One of t h e clear benefits of using a virtual world for this activity is that it gave the task an authenticity that may be sometimes difficult to achieve in language classrooms. As the authors highlight, virtual worlds provide an opportunity for communicative tasks to “be contextualised and made more meaningful”. In this assignment, students had to make their way through a “Chinese island” in Second Life, order correct food in a tearoom and buy ingredients at a market to make dumplings. Through t h e i r evaluation of the assessment task, the authors raise a number of issues relating to t h e assessment of Web 2.0 activities. They question, for instance, the relationship between formative and summative assessment, and suggest that there are opportunities for improvement in the way assessment is managed and in how policy issues are addressed through Web 2.0 assessment tasks. In the next paper, Ru-Chu Shih also describes an assignment that he implemented in the context of a language course, this time involving students in Taiwan who were learning English as a second language. Shih focuses on another Web 2.0 technology that has not, to date, been widely exploited as a learning tool in higher education, Australasian Journal of Educational Technology, 2011, 27(Special issue, 5) vii ePortfolios Australia Conference 2011 (EAC2011) Curtin University, Perth 17-18 October 2011 http://eportfoliosaustralia.word press.com/conference-eac2011/ despite being seen by many to be the epitome of the social web: F a c e b o o k . S h i h demonstrates how Facebook can be useful for supporting students who are learning to write English, enabling students to share and comment on each other’s work. The students in this case study supported each other by providing feedback on their peers’ writing and making suggestions for improvements to grammar and writing clarity. However, Shih notes that students in the lower-ability group would sometimes provide their peers with incorrect suggestions, highlighting a potential challenge of using Web 2.0 to promote peer review. In this case the instructor needed to closely monitor students’ contributions and correct the students’ feedback when they made errors, raising a workload challenge for the instructor. Shih also highlights t h a t the type of feedback enabled by the use of F a c e b o o k includes informal and encouraging feedback, such as the use of emoticons and F a c e b o o k symbols. Drawing largely on student responses to a questionnaire and interviews, the author reports that this assignment was viewed favourably by students, although they also noted challenges with using F a c e b o o k for an English writing assignment, such a s limitations in the number of characters allowed. There were also challenges for t h e instructor in ensuring that the formative feedback provided through F a c e b o o k – which was visible to other students – was encouraging and prompted students to actively participate in the learning task. The paper by John Terrell, Joan Richardson and Margaret Hamilton is, perhaps surprisingly, the only paper in this special issue to report a blogging assignment. For this assignment, students used blogs to report on their experiences of exploring a range of Web 2.0 technologies as information management tools. The student-created blogs formed the assessable component of the assignment, but throughout the activity t h e students used a broad range of Web 2.0 tools. This activity, called a “Web 2.0 adventure”, was undertaken as part of a postgraduate information management course. In this paper, the authors demonstrate that there is a clear link between t h e assignment and the skills currently required of information management professionals. Terrell and colleagues argue that information professionals now need to have a strong understanding of current and emerging technologies, and an awareness of how technologies can be used to support information management. The authors suggest that there is currently a gap between expectations regarding t h e skills and experience of information science graduates, and graduates’ actual Fifth Joint Conference of APACALL and PacCALL De La Salle University Manila, Philippines 27-29 October 2011 http://glocall.org/ viii Australasian Journal of Educational Technology, 2011, 27(Special issue, 5) capabilities with regards to understanding and using Web 2.0 technologies. Drawing on Biggs’ notion of constructive alignment, and through an analysis of students’ blog posts, the authors demonstrate a link between the learning objectives of t h i s assignment and the expectations of “Librarian 2.0” professionals. By interrogating assessment in the context of students undertaking Web 2.0 activities, the authors of papers in this special issue have raised a number of issues that suggest changes in HE teaching and learning that go beyond Web 2.0. They have addressed, for example, assessment for (or even ‘as’) learning rather than assessment of learning; assessment of the learning process rather than the product of learning; reflective learning; collaboration; peer assessment; graduate capabilities; and fading distinctions between summative and formative assessment. This special issue has emerged from our involvement in a project, funded by t h e Australian Learning and Teaching Council, which aims to develop resources to support academics in undertaking assessment of students’ Web 2.0 activities. During this project we conducted 17 case studies focusing on different student assignments in which students were assessed for the work they undertook using Web 2.0 technologies (see Gray et al, 2011, for more information). By publishing this special issue we hoped to both open up discussion about Web 2.0 assessment, inviting others working with Web 2.0 technologies in HE to share their experiences of assessment and research findings, and to provide a vehicle for publishing the case studies from our project. We are pleased, then, that three of the papers in this issue report case studies from our project (Davies et al; Grant and Clerehan; Terrell et al), while three papers report research external to the project (Barwell et al; Ching & Hsu; S h i h ) . Together, these papers contribute to the ongoing dialogue about the opportunities and challenges associated with assessing students’ Web 2.0 activities. References Baltzersen, R. K. (2010). Radical transparency: Open access as a key concept in wiki pedagogy. Australasian Journal of Educational Technology, 26(6), 791-809. http://www.ascilite.org.au/ajet/ajet26/baltzersen.html Elliott, B. (2008). Assessment 2.0: Modernising assessment in the age of Web 2.0. Glasgow: Scottish Qualifications Authority. http://www.scribd.com/doc/461041/Assessment-20 Franklin, T. & van Harmelen, M. (2007). Web 2.0 for content for learning and teaching in higher education. Bristol, UK: JISC. http://ie-repository.jisc.ac.uk/148/1/web2-content-learning- and-teaching.pdf Gray, K., Thompson, C., Sheard, J., Clerehan, R. & Hamilton, M. (2010). Students as Web 2.0 authors: Implications for assessment design and conduct. Australasian Journal of Educational Technology, 26(1), 105-122. http://ascilite.org.au/ajet/ajet26/gray.html Adelaide, 24-25 November 2011. http://www.adelaide.edu.au/clpd/aall2011/ Australasian Journal of Educational Technology, 2011, 27(Special issue, 5) i x Gray, K., Waycott, J., Thompson, C., Clerehan, R., Sheard, J., Hamilton, M. & Richardson, J. (2011). Using social web (Web 2.0) activities for student assessment: Resources for university learning and teaching. http://web2assessmentresources.wikispaces.com/ McLoughlin, C. & Lee, M. J. W. (2007). Social software and participatory learning: Pedagogical choices with technology affordances in the Web 2.0 era. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/mcloughlin.pdf Melville, D., Allan, C., Crampton, J., Fothergill, J., Godfrey, A., Harloe, M., et al (2009). Higher Education in a Web 2.0 World: Report of an independent Committee of Inquiry into the impact on higher education of students’ widespread use of Web 2.0 technologies. http://www.jisc.ac.uk/publications/generalpublications/2009/heweb2.aspx Whitelock, D. (2010). Activating assessment for learning: Are we on the way with web 2.0? In M. J. W. Lee & C. McLoughlin (Eds.), Web 2.0-based-e-learning: Applying social informatics for tertiary teaching (pp. 319-342). Hershey, PA, USA: IGI Global. Editors, Special issue: Dr Jenny Waycott Educational Technology Research Fellow Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Victoria 3010, Australia Dr Judy Sheard Senior Lecturer, Faculty of Information Technology MIT/MAIT Deputy Course Director Co-Director, Computing Education Research Group Director, Monash Museum of Computing History Monash University, Victoria 3145, Australia Please cite as: Waycott, J. & Sheard, J. (2011). Editorial 27(5): Preface to the Special issue. In J. Waycott & J. Sheard (Eds), Assessing students' Web 2.0 activities in higher education. Australasian Journal of Educational Technology, 27(Special issue, 5), iii-ix. http://www.ascilite.org.au/ajet/ajet27/editorial27-5.html Singapore, 6-9 December 2011. http://www.cdtl.nus.edu.sg/tlhe/