Australasian Journal of Educational Technology, 2023, 39(2). 1 The importance of a good review(er) for educational technology research Linda Corrin Deakin University Jason M. Lodge The University of Queensland Kate Thompson Queensland University of Technology The process of peer review has been central to academic publishing in educational technology for at least 50 years. In this editorial we discuss what makes a good review as well as a good reviewer for AJET. This includes an overview of the peer review process and the identification of key features of a good review. We discuss the selection and appointment of reviewers with reference specifically to AJET, and how decisions are made when assigning reviewers to articles. Current challenges facing peer review both broadly in academia and specifically in our field involve the intersection of increasing demand for reviews (due to an expansion of the number of journals), limited opportunities for professional development, and decreasing time available for service to the academy and community for researchers. We conclude with a discussion of the future of peer review practices and how these relate to future directions for AJET. Keywords: educational technology, peer review, educational research Introduction What makes a good review of educational technology research? Furthermore, what are the attributes of a good reviewer of research in this field? As lead editors of an educational technology journal, these are two key questions we revisit often in our ongoing stewardship of the journal. In a publishing landscape that is continually expanding and adapting, the answers to both these questions are increasingly relevant to academic publishers regardless of the academic field. The pressures of institutional requirements to publish and demonstrate impact are becoming important drivers for researchers to share their work with the community faster and wider than ever before (Génova & de la Vara, 2019). This is made easier with many new journals launched by major publishers. More publications means that increasing numbers of researchers need the outcomes of the peer review process to get their work past the publishing post. But in this rush to meet publishing targets and be impactful in their work, do these authors also engage in the process as a reviewer themselves? Some would suggest they do not (Warne, 2016), with the ability to attract good reviewers becoming an increasing challenge for editorial teams. At the same time, new models of peer review are emerging which provide a range of benefits including increased speed, quality, openness, and/or ongoing critique of research outputs. In this editorial we will explore these issues as well as the current state of peer review in the context of educational technology publishing, with a focus on how the peer review process works at the Australasian Journal of Educational Technology (AJET). The history of peer review and common approaches taken by educational technology journals will be discussed including the opportunities and challenges of such approaches. We also consider what makes a good review and what can be done to help develop good peer reviewers in the field. The editorial ends with a look to the future to how peer review can evolve and adapt to continue to serve the purposes for which it has been used for centuries - to ensure that quality research is published and accessible to the field to build knowledge and inform future practice. Australasian Journal of Educational Technology, 2023, 39(2). 2 Why peer review? The process of peer review has been central to academic publishing in educational technology. More broadly, receiving feedback on research outcomes by qualified peers in the field is a tradition dating back to 1731 when the Royal Society of Edinburgh published the first peer-reviewed medical articles (Shema, 2014). However, review of research by peers outside of the lead editorial team took a long time to become standard across academic publishing - with many notable journals not adopting the process of using outside reviewers until after 1940 (e.g., Science in 1940, Nature in 1967, Lancet in 1976) (Tan, 2018). It has been suggested that the introduction of the Xerox photocopier in 1959 greatly helped to accelerate the adoption of peer review once it became easier to replicate manuscripts to send to reviewers (Spier, 2002). By the time educational technology journals came along, such as the British Journal of Educational Technology (BJET) in 1970, Computers & Education in 1976, and AJET in 1985, peer review was a much more commonplace practice in journal publishing. Peer review serves several purposes within academic publishing. It has been said that “the ‘peer- reviewed’ stamp allows readers and the public, as well as funders and other researchers, some degree of currency that the results are strong enough to add to the growing evidence base and turn those insights into the next big … advance” (Siemens, 2022, p.519). Reviewers are tasked with providing feedback about how relevant the research is as well as whether it has been conducted in a manner that would be considered rigorous and within accepted standards for the discipline. Peer review also has an important role within research communities as a mechanism for “how we support each other, how we can push and constructively challenge each other to be truly innovative, … and how we can drive impact and change that are appropriately targeted at the needs of different communities.” (Kohler et al., 2020, p.2). Overall, peer review provides journal editors with an informed recommendation of whether the research publication is of a sufficient level of quality to provide value to the field and be impactful for research and practice into the future and/or what changes need to be made to achieve this goal. What makes a good peer review? For the peer review system to work, it relies on good quality reviews. There isn’t a one-size-fits-all guide to what makes a good review, although there are some key elements that have been found to make reviews useful to authors and recognised as a quality review by editors. In a study of reviews in the journal Limnology and Oceanography: Letters it was found that those reviews that their editors judged to be of high quality tended to be “longer; included more comments related to study goals, analyses conducted, and resulting claims; and contained more descriptive terms related to the manuscript’s importance” (Falkenberg & Soranno, 2018, p.1). Peer reviews should be consistent in the feedback provided and can also highlight the strengths of the article, not just areas for improvement. It is important that peer reviewers offer constructive feedback in a way that is clear, but not overly critical. Clarity of the issues present in the publication and what the authors can do to address them helps to reduce any misunderstanding between the reviewer and the authors. However, in one study of 95 reviewer reports, it was found that many of the suggestions to authors were made in an indirect fashion which authors may find difficult to decode (Paltridge, 2015). While some reviewers adopt a more passive tone so as not to offend the authors, ultimately this could cause unnecessary confusion and result in more time to interpret the true intention of the feedback. Clarity in how the review is structured as well as the suggestion made is important to constructing a good review. It has been observed that there is a current trend for reviewers to be increasingly critical of research they are asked to review (D’Andrea & O’Dwyer, 2017). It has also been observed that bad reviews can enable low-quality articles to get through the process due to the indifference of the reviewers (D’Andrea & O’Dwyer, 2017). A good reviewer should maintain a balanced and open-minded approach to critiquing the work, remain polite in the comments made, and give time to the process to ensure that the comments provided are not the result of only a cursory scan of the article. Editorial teams have a role to play in helping to balance overly critical and apathetic reviews when these may occur, but this can add time to Australasian Journal of Educational Technology, 2023, 39(2). 3 the review process for authors, especially if an additional review is needed to assist in the decision-making process. In understanding what makes a good review, it is useful to contrast this with what is considered to be a bad review. According to Sear (2020) a bad review: includes unprofessional language, is badly structured, doesn’t indicate the level of concerns raised (e.g., major vs. minor revisions), is too brief to be useful for an editor to make a decision, doesn’t justify recommendations made, insists on changes to the research that are unnecessary for the paper under consideration, and makes unfeasible demands for changes. Avoiding these elements should help to ensure that the review is useful to both authors and editors. In addition to research on what makes a good (and bad) peer review, there has also been exploration of what makes a good peer reviewer. A recent international study of what makes for excellent peer review resulted in a proposed competency framework that included three main categories of competencies: (1) foundational knowledge (including expertise, representing, and integrity); (2) mechanical skills (including thoroughness, clarity, and efficiency), and (3) professionalism (including constructiveness, tone, and open-mindedness) (Kohler et al., 2020). These are all key competencies that we seek in AJET reviewers, some of which we explicitly seek in our recruitment process for reviewers and others we aim to develop through professional learning opportunities (see further discussion of these topics below). Challenges to peer review There are a number of challenges that exist to the process of peer review in educational technology journals and the list continues to expand as changes in technology and pressures to publish increase. First and foremost, it is vital that the reviews provided by reviewers are of a high quality. While there are very few published studies that evaluate the quality of reviews, some have explored authors’ perspectives and found that there was a high level of dissatisfaction with reviewer reports received in terms of quality and fairness (Ho et al., 2013). This could be the result of a number of factors that influence the peer review process, including the ways that reviewers are recruited into these roles, as well as the rapid growth in the number of journals in the field which is not allowing the peer review process to function effectively (Altbach & De Wit, 2019). For many years the academic publishing in educational technology research has relied heavily on the availability of peer reviewers. Journals such as AJET make use of a large pool of volunteer reviewers who provide their time and expertise to the journal as part of their service to the academy and the community. However, with the rapid increase in the number of journals in educational technology (and related fields), reviewers are receiving an increasing number of journal review invitations which is reducing their availability. Considering all fields, it is estimated that over 130 million hours was spent on peer reviews of academic articles in 2020, and much of that was done voluntarily (Aczel, Szaszi & Holcombe, 2021). Some reviewers are starting to say no to review invitations if they are dissatisfied with the recognition that they receive for their service in doing the review (Dance, 2023). While others refuse to review for journals that charge authors for publication and turn large profits each financial year without compensating their reviewers (Flaherty, 2022). Workload issues and the need for time to focus on their own research outputs is another key reason quoted for this decline in willingness to review (Dance, 2023). This juggling of competing priorities can create a “bottle neck to the dissemination of time-sensitive material” (Siemens, 2022, p.519), which is especially problematic in the ever evolving field of educational technology. Ideally, authors who submit a paper for review, would themselves volunteer to review other articles. However, at AJET very few of the authors who submit their work to the journal subsequently participate as reviewers. The lack of available peer reviewers contributes to an increase in the time that it can take for an article to move through the peer review process. A recent study of ten computer science journals found that the actual time it takes articles to progress through the peer review process can be up to 500% longer than the average times specified on journal websites (Bilalli, Munir & Abello, 2021). At AJET, Associate Editors are responsible for identifying appropriate reviewers and inviting them to review an article. Recently, we Australasian Journal of Educational Technology, 2023, 39(2). 4 have observed an increase in the number of invitations to undertake a review that are sent in order to receive agreement from two reviewers for the article. For each invitation sent, time needs to be allowed for the reviewer to respond (although, increasingly, many do not), and this delays the review process even further. As an editorial team we are systematically removing those reviewers in our database who repeatedly do not respond to requests for review in order to speed up the review process for authors. A study of reviews for the Journal of Comparative Physiology found that another factor contributing to the amount of time an article spends in the review process is the increasing number of late reviews (Zupanc, 2023). Between 2014 and 2021 they found that the average number of days for late reviewers to submit their reviews almost doubled. An ongoing concern about the peer review process is the lack of reliability between peer reviews. This can be caused by a substantial variability in the quality of the reviews (Kohler et al, 2020), or could be due to the reviewers’ own preferences in terms of research approaches and/or writing style. Sometimes there is a good reason for assigning reviewers who may hold very different perspectives to a paper to ensure that the research design and outcomes will stand up before different audiences in the field. However, the editorial team has an important role to play if two reviews are received with widely differing suggestions to assist the authors in knowing how to respond to such diverse opinions. Another challenge to the peer review process is the increase in fraudulent research that is being submitted for review (Horbach & Halffman, 2019). Peer reviewers can be useful participants in identifying research that may be suspicious. However, the sophistication of questionable submissions is increasing, especially with the new capabilities of artificial intelligence to generate written manuscripts with little input from the researchers themselves (Lodge, Thompson, & Corrin, 2023). Consequently this kind of work is much harder for reviewers to detect. This increase in unethical practices cannot be addressed through the peer review process alone, which is evident by the increasing retractions of published work that had been through peer review across many disciplines (Oranksy, 2022). It has been suggested that other methods, such as the use of technologies to detect academic integrity issues, need to complement peer review in identifying fraudulent work (Horbach & Halffman, 2019). AJET is in the process of implementing the CrossRef tool which will be used to identify academic integrity issues as part of the review process. We have noted that several journals and conferences have also recently focused the questions they ask reviewers to address in their reviews more specific to bias, consent, participant protection, and the merit of the research in an attempt to help reduce the number of questionable articles that get through to publication. This is a piece of work that AJET Lead Editors are currently reviewing in conjunction with a group of international journal editors in the field of educational technology and learning analytics. In addition to the challenge of fraudulent work is the concern that the review process itself may not be unbiased. This bias, sometimes referred to as ‘publication bias’, can be the result of factors such as reviewers only allowing highly innovative papers to be published, the dismissal of replication studies or studies with negative results, and promoting bad methodological decisions like HARKing (i.e., where authors hypothesise retrospectively to fit the data) (Banks et al., 2016; Harford, 2020). Other bias can occur depending on how reviewers are selected, especially when there are undisclosed conflicts of interest. This has led to many calls for greater transparency of the review process to address concerns around fairness and quality (Siemens, 2022). One suggestion is to adopt open peer review processes, which will be discussed in the future directions section below. The integrity of the review process is a matter that we take very seriously at AJET and have been working on ways to monitor and remove any conflicts of interest in relation to the people and processes that make up the peer review cycle. Two years ago we increased the transparency of the first step in the editorial review process by publishing details of the first and second tiers of criteria for inclusion in the journal (see discussion in Corrin et al., 2021). As mentioned above, work on reviewing other elements of the peer review process continues, especially in relation to changing trends in peer review in academic publishing more broadly. Australasian Journal of Educational Technology, 2023, 39(2). 5 The role of reviewers at AJET Without reviewers AJET would not exist. The peer review process is central to maintaining the quality of AJET as well as ensuring that what is published is representative of the interests and priorities of the community. In order to ensure quality reviews of submitted work, there are attributes that a potential reviewer must possess to be offered the role. AJET reviewers usually hold a higher degree (e.g., a PhD or Masters), are research active in the field of educational technology (or closely related field), and have experience of peer review from both the author’s and reviewer’s perspective. When applying to become an AJET reviewer the applicant is asked to provide details of their highest level of qualification, previous reviewing experience for journals and/or conferences, and five of their publications that are most relevant to the AJET context. This information is reviewed by the lead editor team to decide whether to accept the applicant as a reviewer for AJET. Once someone is accepted as a reviewer for AJET they are asked to articulate their key areas of research interest which helps the Associate Editors to identify the most suitable reviewers for each article. Aligning reviewers with relevant experience and interest to articles with particular topics, methodological or theoretical approaches, or analytic techniques helps to ensure that the feedback provided to authors is as useful as possible. Often reviewers will be assigned on the basis of the topic of the research, but we also look to assign reviewers who can provide constructive feedback on other elements such as methodology. This is especially important for research that is highly quantitative in nature, where specialist knowledge of statistical methods is key to being able to comment on the appropriateness and robustness of the approach taken. It is important that the contributions of reviewers are recognised not only by the journals that they support, but also by the institutions in which they are employed. At the completion of the review, the AJET system sends an email to acknowledge the completion and contribution of the review to the review process. Some reviewers use that email to track their reviewing service through web services such as Clarivate’s Web of Science Researcher Profiles (formerly Publons). Reviewers may also include the number of reviews they undertake in performance reviews and/or promotion applications. Journals that are associated with commercial publishers will sometimes offer discounts on purchases of their products, or limited-time free access to papers in the journal as a way of recognising the effort of peer reviewers. As a professional society-funded, open-source journal without article processing fees, the ability of AJET to offer such incentives is limited, but we do continue to consider non-monetary approaches to acknowledge the contributions of our volunteer reviewers. At the annual Australasian Society for Computers in Learning in Tertiary Education (ASCILITE) conference, AJET presents the Distinguished Reviewer Award to a reviewer who has not only completed a number of reviews throughout the year, but also produced high quality reviews in a timely manner. A list of the winners of this award since its inception in 2017 can be found on the ASCILITE website at: https://ascilite.org/awards/ajet-distinguished-reviewer-award/. The AJET Lead Editor team is currently exploring other ways to enhance the recognition of reviewers of the journal, which is only fitting considering the service they provide to the journal and the community. It is hoped that future upgrades to our journal system may enable more automatic and transferable records of reviewer contributions so reviewers can access and use verified records of their activity. How to become a good reviewer Reviewing is often a practice that is picked up along the way, rather than a skill that is intentionally taught and refined. Reviewers can often review an article the way their articles have been reviewed by others. If they have received good reviews by constructively critical reviewers, then this supports the development of good academic practice. However, the opposite can also be the case. Some researchers may have received the benefit of training on reviewing as part of higher degree research studies, but this is not a universal experience. Consequently, the burden of providing professional learning for new reviewers often falls on individual journals, which can be challenging considering the fact that reviewers are already https://ascilite.org/awards/ajet-distinguished-reviewer-award/ Australasian Journal of Educational Technology, 2023, 39(2). 6 giving their time voluntarily to complete reviews and may not have more hours to commit to training and ongoing professional development. As the official journal of ASCILITE, AJET has an important role in academic professional development for authors and reviewers. The team also acknowledges the wealth of experience that many in the community bring to a reviewer role for the journal. For example, as a community that fosters the development of professional staff in roles such as learning designers, educational technologists, etc. those with an established track record of practitioner research in the field, who may not necessarily hold a higher degree, are given the opportunity to review for AJET. The practitioner perspective is one we value, and something that compliments the researcher's perspectives within the review process. It is our intention to develop a program of professional learning to help build reviewer skills in association with the ASCILITE community. While this can help to enhance the quality of peer reviews, it is also helpful to authors considering submitting their work to journals such as AJET. Familiarity with the expectations of the journal and the format in which to present research work, can help to get submissions across the initial editorial review line and into peer review. We are also watching the development of peer review support initiatives in other journals in the higher education context with great interest to see what else might work to support AJET reviewers. For example the Advancing Scholarship and Research in Higher Education (ASRHE) journal operates a group review process involving 5-7 reviewers reviewing research work under the guidance of an editorial team member. This model allows for reviewers to see and discuss the reviews given by others who may be more experienced and to learn from these experiences. Gonzalez et al. (2022) profile several other new approaches to peer review in their recent editorial on the subject, including post-publication peer review, portable peer review, and cascading peer review. Considering how initiatives like these could tap into the ASCILITE community expertise could offer new and exciting opportunities to enhance the peer review process for AJET into the future. The future of peer review in educational technology publishing The future of peer review in the context of educational technology publishing is full of possibilities and challenges. Many academic journals are starting to reconsider the traditional double-blind, closed review model and are exploring other ways to approach peer review. One such model is open peer review (OPR) which can involve publication of the reviews as well as the article. In this model the reviewers may still remain anonymous, or their identities could be published alongside their review. Some OPR journals facilitate the publishing of pre-review manuscripts, and others allow continuing review submissions and comments to be added over time (Wei et al., 2023). While AJET has no immediate plans to introduce an open peer review model to the journal, the editorial team are very supportive of more open research practices in line with the open science movement as discussed in our fourth editorial of 2021 (see Lodge, Corrin, Hwang, & Thompson, 2021). With the rise in use of artificial intelligence within research and publication practices, there are interesting conversations to be had in the educational technology publishing space about how AI can be embraced without undermining the spirit and quality of academic publishing expectations. We have made it clear in our guidelines for the use of AI for AJET that AI is not to be used to generate peer reviews. However, more work is needed to better understand how AI might be used in ethical and useful ways to improve the editorial process. Whether that extends to assistance with peer review in the future is something we will continue to consider as the impact of AI on academic publishing becomes more apparent. What is clear, and has been raised by several commenters in the academic publishing space, is that we need more research into the peer review process in order to ensure its ongoing relevance to academic research dissemination going forward (Resnik & Elmore, 2016; Tennant & Ross-Hellauer, 2020). So much of the data relating to peer review is hidden away from view and rarely studied beyond a single journal. Understanding more about the qualities of reviews, what motivates reviewers to participate in peer reviews, the intricacies of the timelines associated with peer review, and the problems associated with peer reviews would help editors of educational technology journals (and beyond) to put strategies in place Australasian Journal of Educational Technology, 2023, 39(2). 7 to ensure the process can be as ethical, efficient, and impactful as possible. The lead editor team of AJET have begun to work on ways to better use peer review data of the journal for our internal quality assurance purposes, and we will continue to explore ways that we can contribute to the broader conversation in the educational technology publishing context to help maintain high quality publication venues for researchers. Author contributions Linda Corrin: Conceptualisation, Writing - original draft, Writing - review and editing; Jason Lodge: Writing - review and editing; Kate Thompson: Writing – review and editing. Acknowledgements and Appendices This editorial is dedicated to all the reviewers who have helped to make AJET the high quality journal it is today. We value their generosity and time, and hope to continue working together into the future to ensure AJET’s ongoing contributions to impactful research in educational technology. References Aczel, B., Szaszi, B., & Holcombe, A. O. (2021). A billion-dollar donation: estimating the cost of researchers’ time spent on peer review. Research Integrity and Peer Review, 6(1), 1-8. https://doi.org/10.1186/s41073-021-00118-2 Altbach, P. G., & De Wit, H. (2019). Too much academic research is being published. International Higher Education, (96), 2-3. https://doi.org/10.6017/ihe.2019.96.10767 Banks, G. C., Rogelberg, S. G., Woznyj, H. M., Landis, R. S., & Rupp, D. E. (2016). Evidence on questionable research practices: The good, the bad, and the ugly. Journal of Business and Psychology, 31, 323-338. https://doi.org/10.1007/s10869-016-9456-7 Bilalli, B., Munir, R. F., & Abelló, A. (2021). A framework for assessing the peer review duration of journals: Case study in computer science. Scientometrics, 126(1), 545-563. https://doi.org/10.1007/s11192- 020-03742-9 Corrin, L., Hwang, G-J., & Lodge, J. M., Thompson, K., (2021). The role of change in AJET in 2021: reflections, bibliometrics and future plans. Australasian Journal of Educational Technology, 37(6), 1-6. https://doi.org/10.14742/ajet.7721 D’Andrea, R., & O’Dwyer, J. P. (2017). Can editors save peer review from peer reviewers?. PloS one, 12(10), e0186111. https://doi.org/10.1371/journal.pone.0186111 Dance, A. (2023). Stop the peer-review treadmill. I want to get off. Nature, 581-583. https://doi- org/10.1038/d41586-023-00403-8 Falkenberg, L. J., & Soranno, P. A. (2018). Reviewing reviews: An evaluation of peer reviews of journal article submissions. Limnology and Oceanography Bulletin, 27(1), 1-5. https://doi.org/10.1002/lob.10217 Flaherty, C. (2022, June 13). The peer-review crisis. Inside Higher Education. https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals- and-scholars# Génova, G., & de la Vara, J.L. (2019). The Problem Is Not Professional Publishing, But the Publish-or-Perish Culture. Science and Engineering Ethics, 25, 617–619. https://doi.org/10.1007/s11948-017-0015-z Gonzalez, P., Wilson, G. S., & Purvis, A. J. (2022). Peer review in academic publishing: Challenges in achieving the gold standard. Journal of University Teaching & Learning Practice, 19(5). https://ro.uow.edu.au/jutlp/vol19/iss5/01 Harford, T. (2020). How to make the world add up: ten rules for thinking differently about numbers. Hachette UK. Ho, R. C. M., Mak, K. K., Tao, R., Lu, Y., Day, J. R., & Pan, F. (2013). Views on the peer review system of biomedical journals: an online survey of academics from high-ranking universities. BMC medical research methodology, 13, 1-15. https://doi.org/10.1186/1471-2288-13-74 https://doi.org/10.1186/s41073-021-00118-2 https://doi.org/10.6017/ihe.2019.96.10767 https://doi.org/10.1007/s10869-016-9456-7 https://doi.org/10.1007/s11192-020-03742-9 https://doi.org/10.1007/s11192-020-03742-9 https://doi.org/10.14742/ajet.7721 https://doi.org/10.1371/journal.pone.0186111 https://doi-org/10.1038/d41586-023-00403-8 https://doi-org/10.1038/d41586-023-00403-8 https://doi.org/10.1002/lob.10217 https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals-and-scholars https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals-and-scholars https://doi.org/10.1007/s11948-017-0015-z https://ro.uow.edu.au/jutlp/vol19/iss5/01 https://doi.org/10.1186/1471-2288-13-74 Australasian Journal of Educational Technology, 2023, 39(2). 8 Horbach, S. P., & Halffman, W. (2019). The ability of different peer review procedures to flag problematic publications. Scientometrics, 118(1), 339-373. https://doi.org/10.1007/s11192-018-2969-2 Köhler, T., González-Morales, M. G., Banks, G. C., O’Boyle, E. H., Allen, J. A., Sinha, R., ... & Gulick, L. M. (2020). Supporting robust, rigorous, and reliable reviewing as the cornerstone of our profession: Introducing a competency framework for peer review. Industrial and Organizational Psychology, 13(1), 1-27. https://doi.org/10.1017/iop.2019.121 Lodge, J. M., Corrin, L., Hwang, G.-J., & Thompson, K. (2021). Open Science and Educational Technology Research. Australasian Journal of Educational Technology, 37(4), 1–6. https://doi.org/10.14742/ajet.7565 Lodge, J. M., Thompson, K., & Corrin, L. (2023). Mapping out a research agenda for generative artificial intelligence in tertiary education. Australasian Journal of Educational Technology, 39(1), 1–8. https://doi.org/10.14742/ajet.8695 Oransky, I. (2022). Retractions are increasing, but not enough, Nature, 608(7921), 9. https://doi.org/10.1038/d41586-022-02071-6 Paltridge, B. (2015). Referees' comments on submissions to peer-reviewed journals: When is a suggestion not a suggestion?. Studies in higher education, 40(1), 106-122. https://doi.org/10.1080/03075079.2013.818641 Resnik, D. B., & Elmore, S. A. (2016). Ensuring the quality, fairness, and integrity of journal peer review: A possible role of editors. Science and Engineering Ethics, 22, 169-188. https://doi.org/10.1007/s11948- 015-9625-5 Shema, H. (2014). The Birth of Modern Peer Review. Scientific American, 156-160. https://blogs.scientificamerican.com/information-culture/the-birth-of-modern-peer-review/ Siemens, D. R. (2022). Our ongoing evolution of peer review. The Journal of Urology, 208(3), 519-521. https://doi.org/10.1097/JU.0000000000002768 Spier, R. (2002). The history of the peer-review process. Trends in Biotechnology, 20(8), 357-358. https://doi.org/10.1016/S0167-7799(02)01985-6 Tan, M. H. (2018). Peer Review - Past, Present, and Future. In J. Markovac, M. Kleinman, & M. Englesbe (Eds.), Medical and scientific publishing: author, editor, and reviewer perspectives (pp. 55-68). Elsevier Inc. https://doi.org/10.1016/B978-0-12-809969-8.00007-3 Tennant, J. P., & Ross-Hellauer, T. (2020). The limitations to our understanding of peer review. Research integrity and peer review, 5(1), 6. https://doi.org/10.1186/s41073-020-00092-1 Warne, V. (2016). Rewarding reviewers–sense or sensibility? A Wiley study explained. Learned Publishing, 29(1), 41-50. https://doi.org/10.1002/leap.1002 Wei, C., Zhao, J., Ni, J., & Li, J. (2023). What does open peer review bring to scientific articles? Evidence from PLoS journals. Scientometrics, 128(5), 2763-2776. https://doi.org/10.1007/s11192-023-04683-9 Zupanc, G. K. (2023). “It is becoming increasingly difficult to find reviewers”—myths and facts about peer review. Journal of Comparative Physiology A, 1-5. https://doi.org/10.1007/s00359-023-01642-w Corresponding author: Linda Corrin, linda.corrin@deakin.edu.au Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Please cite as: Corrin, L., Lodge, J. M., & Thompson, K. (2023). The importance of a good review(er) for educational technology research. Australasian Journal of Educational Technology, 39(2), 1-8. https://doi.org/10.14742/ajet.8887 https://doi.org/10.1007/s11192-018-2969-2 https://doi.org/10.1017/iop.2019.121 https://doi.org/10.14742/ajet.7565 https://doi.org/10.14742/ajet.8695 https://doi.org/10.1038/d41586-022-02071-6 https://doi.org/10.1080/03075079.2013.818641 https://doi.org/10.1007/s11948-015-9625-5 https://doi.org/10.1007/s11948-015-9625-5 https://blogs.scientificamerican.com/information-culture/the-birth-of-modern-peer-review/ https://doi.org/10.1097/JU.0000000000002768 https://doi.org/10.1016/S0167-7799(02)01985-6 https://doi.org/10.1016/B978-0-12-809969-8.00007-3 https://doi.org/10.1186/s41073-020-00092-1 https://doi.org/10.1002/leap.1002 https://doi.org/10.1007/s11192-023-04683-9 https://doi.org/10.1007/s00359-023-01642-w mailto:linda.corrin@deakin.edu.au https://doi.org/10.14742/ajet.7214 Introduction The role of reviewers at AJET Author contributions Acknowledgements and Appendices References