editorial27-3.pdf Australasian Journal of Educational Technology Volume 27, Number 3, 2011 ISSN 1449-5554 (online) Contents Editorial 27(3): Dawn of a new ERA? ........................................................... iii-viii Tertiary sector Evaluating the impact of educational technology professional development upon adoption of Web 2.0 tools in teaching ................................................... 381-396 Iain Doherty Time advice and learning questions in computer simulations ......................... 397-410 Günter Daniel Rey Learners’ access to tools and experience with technology at the University of the South Pacific: Readiness for e-learning .............................................. 411-427 Shikha Raturi, Robert Hogan and Konai Helu Thaman Collaborative model for remote experimentation laboratories used by non-hierarchical distributed groups of engineering students ......................... 428-445 Oriel A. Herrera and David A. Fuller Learning to be a science teacher: Reflections and lessons from video-based instruction ............................................................................... 446-462 Aik-Ling Tan, Seng-Chee Tan and Marissa Wettasinghe Cooperative or collaborative literacy practices: Mapping metadiscourse in a business students' wiki group project ...................................................... 463-480 Hesham Suleiman Alyousef and Michelle Yvette Picard Data mining techniques for identifying students at risk of failing a computer proficiency test required for graduation ......................................... 481-498 Chih-Fong Tsai, Ching-Tzu Tsai, Chia-Sheng Hung and Po-Sen Hwang Adoption of innovative e-learning support for teaching: A multiple case study at the University of Waikato ............................................................ 499-513 E. Marcia Johnson, Bronwen Cowie, Willem De Lange, Garry Falloon, Craig Hight and Elaine Khoo Schools sector Informing one-to-one computing in primary schools: Student use of netbooks .... 514-530 Kevin Larkin and Glenn Finger E-learning in context: An assessment of student inequalities in a university outreach program ....................................................................... 531-545 Nicole Crawford and Lara McKenzie Integrating book, digital content and robot for enhancing elementary school students’ learning of English ............................................................. 546-561 Nian-Shing Chen, Benazir Quadir and Daniel C. Teng i i Australasian Journal of Educational Technology, 2011, 27(3) The Australasian Journal of Educational Technology (AJET) is a refereed research journal published 6 times per year by the Australasian Society for Computers in Learning in Tertiary Education (ascilite). AJET retired its printed version (ISSN 1449-3098) at the end of Volume 23, 2007, and from Volume 24, 2008, the journal is open access, online only (ISSN 1449-5554), and does not have paid subscriptions. © 2011 Authors retain copyright in their individual articles, whilst copyright in AJET as a compilation is retained by the publisher. Except for authors reproducing their own articles, no part of this journal may be reprinted or reproduced without permission. For further details, and for details on submission of manuscripts and open access to all issues of AJET published since the journal's foundation in 1985, please see http://www.ascilite.org.au/ajet/ For editorial inquiries, contact the Editor, Associate Professor Catherine McLoughlin, School of Education (ACT), Australian Catholic University, PO Box 256, Dickson ACT 2602, Australia. Email: Catherine.McLoughlin@acu.edu.au, Tel: +61 2 6209 1100 Fax +61 2 6209 1185. For review process, production, website and business matters, contact the Production Editor, Dr Roger Atkinson, 5/202 Coode Street, Como WA 6152, Australia. Email: rjatkinson@bigpond.com, Tel: +61 8 9367 1133. Desktop publishing (PDF versions) and HTML by Roger Atkinson. AJET is managed by a Committee comprising ASCILITE Executive nominees, the convenors or nominees from previous ascilite Conferences, and AJET's previous editors and current senior editorial staff. The 2010 Management Committee members are: Professor Mike Keppell, Charles Sturt University, ASCILITE President Dr Philippa Gerbic, Auckland University of Technology, ASCILITE Executive Professor Geoffrey Crisp, University of Adelaide, ASCILITE 2003 Convenor Dr Rob Phillips, Murdoch University, ASCILITE 2004 Convenor Professor Peter Goodyear, University of Sydney, ASCILITE 2006 Convenor Dr Dale Holt, Deakin University, ASCILITE 2008 Convenor Professor Ron Oliver, Edith Cowan University, AJET Editor 1997-2001 Assoc Prof Catherine McLoughlin (Editor), Australian Catholic University Dr Roger Atkinson (Production Editor) AJET's Editorial Board (see http://www.ascilite.org.au/ajet/about/editorial- board.html) reflects the journal's commitment to academic excellence in educational technology and related areas of research and professional practice, our vision of an international journal with an Australasian regional emphasis, and our origins as a professional and learned society publication. Australasian Society for Computers in Learning in Tertiary Education http://www.ascilite.org.au/ Australasian Journal of Educational Technology, 2011, 27(3) iii Editorial 27(3) Dawn of a new ERA? For ERA watchers, the first public sign of a major ‘problem’ with the ARC’s handling of its ERA 2012 Ranked Outlets Consultation was the failure to meet one of its own deadlines. Until about the end of April 2011, the web page for the Review of t h e ERA 2010 Ranked Outlet Lists [1] stated that “A list of the participating peak bodies and academic groups will be made publicly available on the ARC website in mid- April.” Sometime around the end of April or early May, “mid-April” was changed to “end of May”. At the end of May there was no “list of the participating peak bodies”, instead, dated 30 May 2011, something more dramatic appeared: ERA 2012 Journal and Conference Lists The ARC has refined the journal quality indicator for ERA 2012. Profiles of A*, A, B and C ranks will not be used... ... As a result of the above changes, the ARC will not be awarding contracts for ATM 66 - review and provision of recommendations for the ERA 2012 ranked outlets. [2] Some detail about what is meant by “refined the journal quality indicator” appeared in the accompanying press releases from the Minister [3] and the ARC [4]. The Minister’s media release contains about 1018 words in its running text, and one has to read to about word number 816 to get the key point: “I have made the decision to remove the rankings”. So, “refined” means “removed”? However, the preceding text is important reading because it contains evidence of important advances in the ARC’s understanding of matters, especially the recognition “that ERA could work perfectly well without the rankings” [3]. Earlier this year the ARC appeared to hold firmly the view that “The ranked journal and ranked conference lists form an integral part of the ERA evaluation process” [5]. So, within a few months, “integral part” h a s become “work perfectly well without”. Now, h o w and w h y did that happen? An intriguing research question that may be forever unanswered. The ARC’s media release [4] was briefer, about 473 words, and less informative. I t states that the ARC “will use a refined journal quality indicator for ERA 2012”, without giving any tangible information about how its next attempt to refine will be better than its previous attempts. The media release reiterates the ARC’s view t h a t “journal quality is an important indicator of research quality”, without referring to the core problem, the goodness (or poorness) of correlation between “journal quality” and the “research quality” of individual articles that a journal has published. Hobart, 4-7 December 2011. http://www.ascilite.org.au/conferences/hobart11/ iv Australasian Journal of Educational Technology, 2011, 27(3) HERDSA 2011 Higher education on the edge Gold Coast, Queensland 4-7 July 2011 http://conference.herdsa.org.au/2011/ Media outlets very quickly characterised the Australian research community’s general reaction to the Ministerial and ARC press releases: Kim Carr bows to rank rebellion over journal rankings Relief and a sense of vindication over its stand against journal rankings has characterised the research community's reaction to Innovation Minister Kim Carr's surprise dumping of the measure that made the government's research evaluation process so controversial. [6] Journal rankings abolished - and unis couldn't be happier After months of complaints, Kim Carr has axed the controversial ERA journal rankings. Universities have welcomed an unexpected announcement by Research Minister Kim Carr to abolish journal rankings... [7] Protests force research policy change Widespread complaints about how Australian academic research is assessed has led to the scrapping of the most controversial aspect of the new assessment scheme: ranking academic journals. [8] Troubled history of an ERA In what must be an embarrassing backdown for the Australian Research Council and the Minister for Science and Research, Senator Kim Carr, one element of the controversial Excellence in Research for Australia assessment scheme has been scrapped: the scheme's method of ranking academic journals which had been roundly criticised from the moment anyone started to take notice of it. [9] The Ministerial media release [3] contains several discordant passages that could be regarded by some as ‘blame shifting’. Firstly, there is reference to the previous government and to the bibliometric advice that the ARC commissioned: ... the rankings themselves were inherited from the discontinued Research Quality Framework (RQF) process of the previous government, and were developed on the basis of expert bibliometric advice. [3] Secondly, “institutional research managers” seem to be implicated for “ill-informed, undesirable behaviour in the management of research”: There is clear and consistent evidence that the rankings were being deployed inappropriately within some quarters of the sector, in ways that could produce harmful outcomes, and based on a poor understanding of the actual role of the rankings. One common example was the setting of targets for publication in A and A* journals by institutional research managers. [3] So, the Tiers ranking was fine, but “ill-informed, undesirable behaviour in the management of research” has ‘forced’ the Minister to make “the decision to remove the rankings, based on the ARC’s expert advice.” Not a good ‘rally the troops’ call! IEEE International Conference on Technology for Education (T4E) Chennai, India, 14-16 July 2011 http://www.tfore.iitm.ac.in/ Australasian Journal of Educational Technology, 2011, 27(3) v Moodlemoot AU 2011 Sydney Convention and Exhibition Centre, 17-20 July http://moodlemoot.org.au/ The Ministerial media release [3] promises “The introduction of a journal quality profile, showing the most frequently published journals for each unit of evaluation”. At first guess it seemed that “most frequently published” should read “most frequently cited”, leading to the speculation that “journal quality profile” w i l l utilise the Thomson Reuters Impact Factor [10], or one of its competitors such a s Elsevier’s SNIP and SJR [11]. A partial clarification appeared on about 3 June 2011 in the Frequently Asked Questions page [12]: What is the change to the journal indicator? During the ERA 2012 evaluation, REC [Research Evaluation Committee] members will be provided with the new journal indicator, which will replace the ranked journal indicator used in the ERA 2010 evaluation. The new indicator will display, for each unit of evaluation, a table listing the journals in which the articles submitted to that UoE [Unit of Evaluation] are published ... The table will inform expert judgements regarding the relevance of the journals to the research being published e.g. ‘Is this an appropriate journal for this research?’. ‘Is it a highly regarded journal?’ This will allow RECs to take into account any regional or applied focus of research in a UoE. As before, REC members will be able to drill down to article level data from the table, so will not be making their judgments solely on the basis of journal titles or article counts. The table will not include information about the quality of journals... The new journal indicator will not use prescriptive A*/A/B/C ranks. [12] It seems to be a partial clarification, as it contains a possibly ominous sentence, ‘Is i t a highly regarded journal?’ We will have to wait and see the definitions, if any ever appear, for ‘highly regarded journal’, but we do note that the ARC is seeking tenders closing 16 June 2011 for “ATM 75 Provision of bibliometric information for ERA 2012” [13]. ATM 75 is dated 12 May 2011 and clearly was prepared and placed into distribution before the “the decision to remove the rankings”. ATM is accompanied by ‘Addendum 2 June 2011.pdf’ [13] and its purpose was to update ATM 75 by removing various references to “discipline-specific tiered outlet rankings”, and point out in several places that “Outlets will not be ranked in ERA 2012”. Nevertheless, one reference to ranked outlets remained in clause 3.3.4: Journal article and conference publication coverage will be assessed on the ERA 2010 ranked outlets lists available from http://www.arc.gov.au/era/key_docs10.htm [13] However, the revised version of http://www.arc.gov.au/era/key_docs10.htm h a s an important new proviso, “Please note that material on this page is strictly related Association for Learning Technology: ALT-C 2011 Thriving in a colder and more challenging climate University of Leeds, UK, 6-8 September 2011 http://www.alt.ac.uk/altc2011/ vi Australasian Journal of Educational Technology, 2011, 27(3) 5th Asia Pacific Conference on Educational Integrity The University of Western Australia Perth, 26-28 September 2011 http://www.catl.uwa.edu.au/5apcei to the ERA 2010 process and is not relevant for the ERA 2012 process.” The documents suggest that the critical decision meeting to “to remove the rankings” was on some date between about 13 May and 29 May 2011. Thus in a ‘5 Ws’ [14] approach, we can estimate a when, though there’s little prospect of progress with the other Ws. Most notably, on the why, a rather wide gap is obvious between the ARC-Ministerial perspective and the media commentary quoted above. To illustrate further the w h y from the ARC’s perspective, the current Frequently Asked Questions page [12] states: Why are ranks no longer being used? The change enables journal quality to remain an indicator for ERA 2012, while discouraging the use of assessments of journal quality beyond their role as an ERA indicator. It will ensure that the indicator is kept in proper perspective, while maintaining ERA’s rigour and focus on quality. An essential feature of ERA is and has always been that it uses a range of indicators—not just journal rankings. Those most engaged with ERA understand this, however, the focus on journal rankings has led to some undesirable consequences. Experience from the ERA 2010 evaluation meeting also suggested that, although journal quality is an important indicator, the ranks themselves did not play a crucial part in the evaluation process. [12] So “work perfectly well without” in the Ministerial media release [3] has become “did not play a crucial part” and “journal quality to remain an indicator”. What w i l l be the next morphing? Note again a somewhat supercilious implication that critics do not understand: “Those most engaged with ERA understand this ...”. However, the lack of information about what really happened, and lack of d e t a i l about the Tiers replacement, the new ”journal quality profile”, does have a brighter side from a journalism perspective. If there are few facts, creative writing may be unleashed, as in the witty, satirical speculation by Joseph Gora [15]: Inside the ERA bunker WIKIFREAKS has obtained a record of the following exchange that took place deep in the Australian Research Council bunker on the northern front of the Battle of the Journals' Ranking System. [15] Where to now? No doubt many editors of journals will make cautious adjustments to their ‘about this journal’ pages. Cautious, because there will be uncertainty about this new unknown, ”journal quality profile”, and because the ARC has not ‘un- published’ the Tiers 2010 list [16], in contrast to its swift removal of the Tiers 2008 list. In AJET’s case, we will work upon a sentence that states (in an academic way), that AJET was A in Tiers 2008, was demoted to B in Tiers 2010, and now is free from that artificial, arbitrary and counter-productive shackle. In relation to Editorial advice to readers, authors and Society members, again a cautious line seems appropriate, because some previous predictions didn’t happen. In Editorial 27(1) we speculated that: Australasian Journal of Educational Technology, 2011, 27(3) vii Nusa Dua, Bali, Indonesia, 2-5 October 2011. http://www.ut.ac.id/icde2011/ The ARC may intend to ease the pressure a little by softening its stance on the rigidly normative nature of Tiers (5% A*, 15% A, 30% B, 50% C).... We note that references to 5% A*... seem to have disappeared from the ERA website, or have become deeply buried. Could we be on the verge of something less severely normative, e.g. 10% A*, 20% A, 3 5 % B, 35% C? [17] So, a wee bit off the mark there, normative is apparently abandoned, but we w i l l repeat another RA prediction, made in Editorial 27(2): The next big emerging topic for research into research management and leadership policies may be the topic of tensions between the "rewarding of excellence", and the "rewarding of research that aligns well with academic, community and Government priorities". [18] However, as ‘refinement’ is in vogue, let’s refine "rewarding of research that aligns well with academic, community and Government priorities" into rewarding o f excellence that aligns well with academic, community and Government r e s e a r c h priorities. One could expect that Government (some would add, especially a L a b o r Government) will be happier with that direction, compared with the demonstrably narrow direction fed to it by the current leadership of the ARC. The ARC seems to have fallen into the trap of becoming a petty bureacratic, controlling agency, losing any scope for being an influential and visionary contributor to academic research leadership in Australia and beyond. Roger Atkinson and Catherine McLoughlin AJET Production Editor and AJET Editor Endnotes 1. ARC (2011). Review of the ERA 2010 Ranked Outlet Lists. http://www.arc.gov.au/era/era_2012/review_of_era10_ranked_outlet_lists.htm 2. The document ATM 66 detailed tender requirements for those seeking to become one of the “participating peak bodies”. This document has been ‘unpublished’, but if searching for a copy, the original filename was ‘ERA_2012_ranked_outlets.pdf’. 3. Carr, Senator the Hon Kim (2011). Improvements to Excellence in Research for Australia. Media release, 30 May. http://minister.innovation.gov.au/Carr/MediaReleases/ Pages/IMPROVEMENTSTOEXCELLENCEINRESEARCHFORAUSTRALIA.aspx 4. ARC (2011). Excellence in Research for Australia 2012. Media release, 30 May. http://www.arc.gov.au/media/releases/media_30May11.htm ePortfolios Australia Conference 2011 (EAC2011) Curtin University, Perth 17-18 October 2011 http://eportfoliosaustralia.word press.com/conference-eac2011/ viii Australasian Journal of Educational Technology, 2011, 27(3) 5. This sentence was the first in reference [1] until about 30 May 2011. Now it has been ‘unpublished’, however records will continue to exist elsewhere, my own being in Atkinson, R. J. (2011). Technology and control of feedback: My encounters with ROCI. HERDSA News, 33(1). http://www.roger-atkinson.id.au/pubs/herdsa-news/33-1.html 6. Rowbotham, J. (2011). Kim Carr bows to rank rebellion over journal rankings. The Australian, 1 June. http://www.theaustralian.com.au/higher-education/kim-carr-bows-to- rank-rebellion/story-e6frgcjx-1226066727078 7. Woodward, S. (2011). Journal rankings abolished - and unis couldn't be happier. Campus Review, 6 June. http://www.campusreview.com.au/pages/section/article.php?s=News&idArticle=21128 8. Maslen, G. (2011). Protests force research policy change. University World News, 174, 5 June 2011. http://www.universityworldnews.com/article.php?story=20110603184401784 9. Dobson, I. (2011). Troubled history of an ERA. University World News, 174, 5 June 2011 10. Thomson Reuters (2011). The Thomson Reuters Impact Factor. http://thomsonreuters.com/products_services/science/free/essays/impact_factor/ 11. Elsevier B. V. (2010). SNIP & SJR: A new perspective in journal metrics. http://info.scopus.com/journalmetrics/ 12. ARC (2011). Frequently asked questions: ERA 2012. [viewed 11 Jun 2011] http://www.arc.gov.au/era/faq.htm 13. ARC (2011). Tenders. ATM75 Provision of bibliometric information for ERA 2012. http://www.arc.gov.au/about_arc/tenders.htm. Filenames ‘Frequently Asked Questions ATM 75 3 June 2011.pdf’, ‘Addendum 2 June 2011.pdf’ and ‘Approach to Market (ATM) 75 - Citation provider 2012.pdf’ 14. Wikipedia. Five Ws. http://en.wikipedia.org/wiki/Five_Ws 15. Gora, J. (2011). Inside the ERA bunker. The Australian, 8 June. 16. ARC (2010). http://www.arc.gov.au/xls/ERA2010_journal_title_list.xls (Excel format, 5.27 MB, apparently dated 9 February 2010). For those who find the ARC's file too cumbersome and slow to handle on their personal computers, we recommend Lamp, J. (2010). ERA Current Rankings Access. http://lamp.infosys.deakin.edu.au/era/. See also John Lamp’s bibliography, Publications on ERA Outlet Ranking, http://lamp.infosys.deakin.edu.au/era/?page=pubs 17. AJET Editorial 27(1). Review of the ERA 2010 Ranked Outlet Lists http://www.ascilite.org.au/ajet/ajet27/editorial27-1.html 18. AJET Editorial 27(2). Educational sector representation in educational technology journals http://www.ascilite.org.au/ajet/ajet27/editorial27-2.html Fifth Joint Conference of APACALL and PacCALL De La Salle University Manila, Philippines 27-29 October 2011 http://glocall.org/ Adelaide, 24-25 November 2011. http://www.adelaide.edu.au/clpd/aall2011/ Singapore, 6-9 December 2011. http://www.cdtl.nus.edu.sg/tlhe/