Australasian Journal of Educational Technology, 2021, 37(3). 104 How and why are digital badges being used in higher education in New Zealand? Maggie Hartnett Massey University Digital technologies, as mediators and facilitators of learning, are altering tertiary education; how and when it occurs, what it entails, who has access, and how capabilities and skills are acknowledged. Digital badges are one such technological tool. Created to acknowledge competency, skill or achievement they have been adopted for a variety of purposes including to motivate learners, recognise achievement and accredit learning. Internationally, the use of digital badges is growing; however, much of the existing literature addresses the potential of digital badges while there is a relative paucity of empirical research, particularly in the Australasian region. This research explored the use of digital badges within New Zealand’s public higher education sector. Using a mixed methods approach (national survey of staff followed by interviews), results revealed over half of the institutions were using badges or planned to in the future. Identified benefits included displaying achievement, motivating learners and evidencing learning. Challenges were also noted, including faculty members’ lack of knowledge about badges, inconsistent use and lack of formal regulation of badges. The findings suggest that badge use is at the early adoption stage and provide valuable insights from which to develop future practice. Implications for policy or practice: • Decision-makers need to be mindful that organisational culture considerations can affect the success of badging systems and micro-credentials. • Implementers of digital badging systems need to ensure staff understand the value and credibility of digital badges. • Staff responsible for awarding digital badges need to be involved in decisions about how and why they are used beyond individual units of study. Keywords: digital badges, micro-credentials, adoption, higher education, tertiary, learning, New Zealand Introduction Digital technologies are fundamentally changing the accessibility, organisation and credentialling of education. Digital badges are a technological tool developed to acknowledge competency, skill or achievement. They are used for various purposes including motivation, achievement and accreditation. Digital badges are “a representation of an accomplishment, interest or affiliation that is visual, available online, and contains metadata including links that help explain the context, meaning, process and result of an activity” (Gibson et al., 2013, p. 404). They offer the ability to acknowledge achievements at a fine- grained (granular) level recognising and credentialling the achievement of specific competencies, knowledge and skills (Peck et al., 2016). Digital badges, considered a disruptive technological innovation (Lockley et al., 2016), promise new opportunities for engaging and motivating learners, implementing new assessment approaches that recognise alternative achievement, tracking learning progress, scaffolding learning, recognise learning, encouraging behaviours that contribute towards learning and sharing accomplishments (Ahn et al., 2014; Devedžić & Jovanović, 2015; Gibson et al., 2013). Badges can augment current formal accreditation structures (Reid et al., 2015). However, it is opportunities to offer alternatives to traditional academic transcripts by recognising non-formal, informal and professional learning accomplishments, using metadata related to the accomplishment, which offer new possibilities for learning (Fields, 2015). Digital badges, for example, can be used to recognise and credential meta-skills such as critical thinking, collaboration or communication at the granular level (Ahn et al., 2014; Australasian Journal of Educational Technology, 2021, 37(3). 105 Finkelstein et al., 2013). Badges can offer more flexible learning pathways and lifelong learning opportunities for learners (Lockley et al., 2016). Badges can be used as pedagogical tools, for example, by linking badges to specific learning outcomes of a subject or unit of study, enabling learners (and faculty) to track learning progress (Kehoe & Goudzwaard, 2015). Badges also enable recipients to decide in which virtual sites (e.g., social or professional networking sites) they wish to share their achievements (O'Byrne et al., 2015). Depending on the granularity of the badges, a hierarchy of badges can be developed where more major badges recognise higher attainment and can lead to more substantial credentials often called micro-credentials (Ellis et al., 2016). The terms digital badges and micro-credentials are used interchangeably, which can cause confusion. Oliver (2019) defined a micro-credential as “a certification of assessed learning that is additional, alternate, complementary to or a formal component of a formal qualification” (p. i). An alternative understanding is where digital badges are visual tokens used to acknowledge learning and achievement that can then be aggregated to represent micro-credentials depending on the purpose and context. This is the view adopted in this research. The potential of digital badges as motivators has been identified by several researchers (e.g., Abramovich & Wardrip, 2016; Berge & Muilenburg, 2016; Ellis et al., 2016; Gibson et al., 2013). Research shows that when learners see digital badges as valuable, they can positively influence learning (Facey-Shaw et al., 2018; Hakulinen et al., 2015). However, the use of badges for motivational purposes is not straightforward as the badge type, the prior knowledge of the learner, the expectation the learner has of learning and the value they place on that learning all influence how badges are perceived (Abramovich & Wardrip, 2016; Reid et al., 2015). As with other new technologies that offer the potential for fundamental change, badges are not without issues. The success of digital badges depends on their perceived value to stakeholders. For badges to have value as credentials, and therefore be trusted, they need to be valid and verifiable. Therefore, the quality of evidence associated with the badge is a fundamental consideration (Ellis et al., 2016). Given any organisation can develop and implement their own badging system independent of other groups who are likely to have an interest in them, such as schools, employers, other learning organisations and educational policymakers, creating valued and verifiable badging systems is not straightforward (Grant, 2016). At the institutional level, implementing badging systems is likely to require major restructuring of institutional strategies, processes and cultural practices (Lockley et al., 2016; Oliver, 2019). Due to the complexity of implementation, pilot studies within individual institutions (particularly within higher education) have occurred at the subject or course level (Lockley et al., 2016). In addition to the challenges outlined above, the reluctance of some faculty to engage with digital badging due to lack of time, incentive and knowledge of a range of assessment practices (e.g., competency-based assessment) has been noted (Lockley et al., 2016; Peck et al., 2016). Sceptics also raise concerns about the number and types of badges available, highlighting that multiple badges may result in fragmented approaches to subject or content areas (Peck et al., 2016). Integrating badges into learning experiences, rather than using them as add-ons, is one strategy used to address concerns and ensure badges are more than tokens (Abramovich, 2016; Casilli & Hickey, 2016). The adoption of digital badges is growing internationally throughout higher education (Grant, 2016). However, ensuring tertiary education digital badging or credentialling systems function effectively at both national and international levels is complex, takes time to develop and involves multiple actors. Groups with a stake in the development of national digital credential systems include learners and employers who use the system; academic institutions who award the digital credentials and digital organisations who create the platforms on which digital credentialling systems operate; government agencies, qualifications authorities and standards bodies who quality-assure the credentials; and qualification agencies and government departments who verify qualifications (Chakroun & Keevy, 2018). Currently, various initiatives are underway to develop the foundational systems necessary to support digital badges and micro- credentials both nationally and internationally. At an international level, the Groningen Declaration Network has focused on bringing together key stakeholders to enable the development of digital student data portability (Groningen Declaration, n.d.). The European Commission’s goal is to develop the European Education Area by 2025, which seeks to remove political and technical obstacles that currently prevent the recognition of skills and qualifications, acquired across the lifespan, throughout the European Union (European Commission, 2017). Several countries are also making progress towards such systems. For example, Korea, Singapore and China have developed digital systems enabling their citizens to register Australasian Journal of Educational Technology, 2021, 37(3). 106 their credentials as well as learning credits. Progress in providing digital access to certified qualifications to interested parties via My eQuals has occurred in Australia and New Zealand (Oliver, 2019), and the New Zealand Qualifications Authority (n.d.) has formally recognised micro-credentials within the national qualification framework. Recent developments in the United States of America and Canada have seen the creation of qualification frameworks that are more inclusive of non-degree credentials while also incorporating the digitisation of credentials (Keevy & Chakroun, 2015). While work continues to develop the necessary frameworks outlined above, current studies suggest digital badge usage within tertiary education varies widely and includes badge use within stand-alone subjects or units (micro level) to encourage learner motivation and indicate progress, badging systems at the programme (meso) level to recognise meta-skills such as critical thinking and digital literacy, through to the utilisation of complete badging systems for official recognition at the macro (i.e., institutional) level. However, it is unclear which institutions are using digital badges and how they are using them (Grant, 2016) as there is limited empirical research (Carey & Stefaniak, 2018; Lockley et al., 2016; Roy & Clark, 2019) particularly in Australia and New Zealand (Selvaratnam & Sankey, 2019). Therefore, further research is needed to develop digital badge practice, policy and theory in higher education. Methodology This project centred on the use of digital badges in public higher education institutions within a single jurisdiction, in this case New Zealand. In doing so, it seeks to explore how and why digital badges are being adopted and used to promote, support and recognise learning. This investigation used a mixed methods research methodology. Proponents (e.g., Bazeley, 2018; Creswell & Plano Clark, 2011) have argued that the combination of both quantitative and qualitative methods “provides a stronger understanding of the research problem or question than either by itself” (Creswell, 2014, p. 215). There are various mixed methods designs, including explanatory, exploratory and embedded (Creswell & Plano Clark, 2011). An explanatory sequential design (Creswell & Plano Clark, 2011) was adopted, where Phase 1 involved a national survey of current digital badge use within New Zealand public tertiary institutions and Phase 2 complemented and enhanced the survey data set with a series of follow-up interviews with a subset of survey respondents. The research question that guided this investigation was “How are digital badges being adopted and used in public higher education institutions in New Zealand”? Data collection Prior to the collection of data, ethical consent to undertake the study was obtained in accordance with the ethical standards of Massey University’s Human Ethics Committee. Informed consent was obtained from all individual participants included in the study. The focus of the phase one was to identify current digital badge development, use and implementation (i.e., which higher education institutions were using badges, and the perceived benefits and drawbacks associated with their use) within the sector. The relatively small number of public higher education institutions within New Zealand (27) made it possible to investigate digital badge use across institutions that operate within the same policy environment. An anonymous online survey was distributed to staff in all 27 public tertiary institutions encompassing eight universities, 16 institutes of technology and polytechnics (ITPs) and three wānanga (a tertiary institution that provides education in a Māori cultural context). The survey questions were developed with reference to the literature associated with digital badges and comprised two sections. The first section consisted of demographic questions of an individual (e.g., age, gender) and professional nature (e.g., qualifications, professional role, employing institution). The second section asked questions related to the respondents’ knowledge and use of digital badges. The final survey question asked respondents if they would be interested in taking part in a follow-up interview. For those respondents who self-identified for phase two, their survey responses were anonymised prior to analysis. The survey was hosted via SurveyMonkey (https://surveymonkey.com) and was piloted prior to distribution. Phase 2 of the project consisted of semi-structured interviews with a subset of survey respondents who agreed to take part in the follow-up phase of the project. Interview questions were developed to complement and expand on survey findings and included several sections: knowledge and experience of digital badges, benefits and challenges associated with digital badges, support received associated with digital badge development and use and criteria for successful digital badge adoption. The interviews were a maximum https://surveymonkey.com/ Australasian Journal of Educational Technology, 2021, 37(3). 107 of 1-hour duration and occurred predominantly via Skype or similar videoconferencing technology. Face- to-face interviews were conducted where practicable if a participant preferred. The interviews were digitally recorded with the permission of the participants. Analysis of the quantitative survey data involved the calculation of summary statistics. Analysis of qualitative data involved reading and re-reading all interview responses to determine the breadth of themes and sub-themes. Responses were coded to one or more sub-themes depending on whether single or multiple ideas were identified. The qualitative analysis software package NVivo (https://www.qsrinternational.com) was used to facilitate this process. Participants Intended survey participants were academic and support staff working in public institutions who had some knowledge or experience of the implementation and/or use of digital badges as part of their professional role. Late in 2017, an invitation to participate in the survey was distributed, via email, to key people within each institution (e.g., digital learning managers or directors, academic and professional development managers, directors of teaching and learning and academic development staff) and a request made for them to disseminate the invitation to staff within their organisation. The email contained a direct link to the survey. To ensure a broad representation of individual and institutional views, informal networks were also used for survey dissemination. These included my contacts with professional networks and social media channels such as Twitter and Facebook. A total of 124 survey responses were received from self-selecting respondents working in 24 of the 27 New Zealand public tertiary institutions. Staff from two of the wānanga and one of the ITPs did not respond to the survey. A total of 14 of the survey respondents indicated they were interested in taking part in a follow-up interview: eight from the university sector and six from the ITP sector. All 14 volunteers were interviewed in early 2018, of whom five were academic developers, five were faculty, three held leadership positions and one was a technical support person. Pseudonyms are used when reporting interview findings. Findings Survey demographics All results report valid percentages (i.e., missing responses are excluded). Of the responses received, 58% (69/119) came from the university sector, 37% (44/119) from the ITP sector and 1% (1/119) from wānanga. The remaining 4% (5/119) of responses consisted of people who did not identify with a specific institution because they worked for an organisation affiliated with the tertiary education sector more generally (e.g., Ako Aotearoa, a government-funded organisation that supports New Zealand’s tertiary sector educators). Comparison with New Zealand Ministry of Education (2017) workforce figures show that university staff were slightly under-represented in the survey (58% versus 69% of the tertiary sector workforce); ITP staff were slightly over-represented (37% versus 26%) and wānanga were under-represented (1% versus 5%). Respondents worked in a range of academic disciplines; education, business, health, foundation studies, science, humanities and social sciences, and arts and design were the prevalent. The majority (90%, 111/124) of respondents listed their age as 40 years old or above. The gender mix of the group was 60% female (75/124), 39% male (48/124) and 1% (1/124) preferred not to say. Respondents were asked to identify their highest qualification, and results indicated that 91% (113/124) had completed a postgraduate qualification ranging from a postgraduate certificate or diploma (18%), master’s degree (45%) to a doctorate (28%). Participants were also asked about their current professional role (see Table 1). Table 1 shows that most respondents held academic (i.e., faculty), academic development (i.e., faculty support), leadership or teaching support roles. Participants for whom teaching was part of their role were also asked about their years of teaching experience. Of the respondents who indicated they were teaching, 95% (81/85) had experience of 10 years or more. https://www.qsrinternational.com/ Australasian Journal of Educational Technology, 2021, 37(3). 108 Table 1 Current professional role of survey respondents Role n % Faculty 40 32.3% Academic development 33 26.6% Leadership 22 17.7% Teaching support 18 14.5% Technical support 4 3.2% Other 7 5.6% Total 124 100% Benefits of digital badge use All survey respondents were asked to indicate (choosing all statements that applied) what value digital badges offered (see Figure 1). The respondents (n = 110) chose a total of 705 statements. Over half of the respondents rated the following, in descending order, as valuable aspects of digital badges: • a display of achievement • a motivational aid for learners • digital evidence of learning • encouragement of participation • recognition of informal learning. Only four people indicated they had no value. Of the 18 who chose “other”, responses included “all OERu micro-credentials are mapped to official academic credit”, “immediacy of feedback & recognition” and “allows recognition of contributions to community/society” suggesting badges have value at both micro and macro levels. Similar themes related to the benefits of digital badges emerged from follow-up interviews. Motivation was the most salient benefit identified by all interviewees. The comment from Henry (ITP, faculty) represents those made by all participants, namely “They can be motivating forces. So, if you use badges to recognise completion or achievement of certain modules or units in the course [subject], then … they can be really good.” Following on from motivation, one of the noticeable benefits that interviewees identified was the tangible and visible nature of digital badges as a display of achievement. For example, Ian (university, leadership) highlighted badges as “that tangible form of acknowledgement or tangible acknowledgement of those competencies”, while Kyle (university, academic development) pointed to the “visual recognition of learning … something visual for the students”. The potential of badges to recognise various forms of learning was something 12 out of the 14 interviewees commented on. For example, Grace (university, faculty) stated that “for me it is one way of acknowledging their work immediately, so they know somewhere my work has been registered and here is the feedback.” Ari (university, faculty) shared how his students appreciated the opportunity to gain recognition for their work, “when I mentioned to them [the students] that they can get this badge, they jump[ed] on that idea, like oh, ‘I can get something for that?’ Some recognition and acknowledgement of it.” Seven out of the 14 interviewees also identified the value of digital badges as a source of evidence, such as “I think the key is the evidence for the learning with the digital badges” (Oliver, ITP, faculty) and “the reason I understood was, it was a way … that you can show professionally, especially in the evidence kind of based society, prove that you’ve done this” (Emily, ITP, teaching support). Australasian Journal of Educational Technology, 2021, 37(3). 109 Figure 1. Perceived value of digital badges Some benefits of digital badges featured more strongly in the interview findings than the survey results. Digital badges as support for learning in terms of tracking progress (for both faculty and students) was a more salient finding from the interviews, with nine interview participants (64%) identifying this as an important value of digital badges, whereas only 20/110 survey respondents (18%) indicated it was a benefit. As Daniel (university, faculty) highlighted, “I guess from an assessment point of view, if you micro- credential [badge] particular aspects of a larger set of learning it helps people map their progress against that broader package of learning.” Anna (university, leadership) also mentioned the importance of learners being able to track their own progress, “for students [it is] tracking themselves to a goal, being able to see that pathway and to see exactly where they are on that pathway.” The importance of digital badges being easily shared was a further benefit identified by nine out of the 14 (64%) interview participants. For example, Anthony (ITP, teaching support) mentioned that “being able to show it off to others is simply a way… either to show that I am amazing or that I can help you because I am an expert in this” was an important advantage of digital badges. Henry (ITP, teaching support) also commented on the value of learners being able to share their digital badges: “the students could put the badges that they achieved on their profile page on the learning management system”. Challenges of digital badge use All survey respondents were also asked what they perceived to be the main challenges of digital badge use (see Figure 2). Respondents (n = 109) chose a total of 353 statements. Respondents rated the following as the main challenges of digital badges: lack of knowledge (54/109), inconsistent use (53/109), and lack of formal recognition (49/109). Additionally, lack of regulation (43/109), lack of wide recognition (43/109) (e.g., by employers) and lack of training (42/109) were concerns identified by over one third of respondents. Just under 5% (5/109) of participants saw no drawbacks associated with digital badges. Of the 18% (20/109) who chose the “other” category, responses included the following: 0% 10% 20% 30% 40% 50% 60% 70% 80% Australasian Journal of Educational Technology, 2021, 37(3). 110 Some teachers have never heard of e-badges let alone considering implementing it into their teaching. In fact, when I raised this possibility of recognition of soft skills (in the Faculty of Science) some mockingly said, “why not display those on the lab coat?” (Respondent 12) Badges as used on quizzes as 1:1 analogues for score. Other poor uses that put students off, as they see through the ‘game’ mechanism as manipulation. That's a sure sign of poor implementation. (Respondent 88) Figure 2. Main challenges to digital badge use Interview participants highlighted several similar themes regarding the challenges associated with digital badge use. The most frequent theme identified was the limited understanding or lack of knowledge (at the faculty and institutional level) about digital badges. The comments from William (university, academic development) and Oliver (ITP, faculty) respectively represent 11 of the 14 interviewees when they say “confusion of what they’re [badges] about –the implementation of them, because it will take work” and “I just think it’s at a very early stage and that’s not really properly understood. I don’t think they’re understood, as such.” Inconsistent and superficial use of digital badges was also highlighted by 12 of the 14 interview participants. Natalie (ITP, faculty) talked about the importance of learners understanding and valuing badges in the following: For students, it would be ‘I can’t see the point, what’s this doing’? … Or they might get overused in the programme, they’re too easy to get, so ‘I’ve got badges, so what’? So, after 6 months they might go ‘well that’s just a waste of time’. Thomas (ITP, leadership) cautioned against the overuse of badges: I think that the way that badges could be used if rolled out en masse will lead to the degradation of the inherent value of a badge because for instance, if I am storing it in a backpack and I have a million of them; literally a million of them, they are all worthless. Anthony (ITP, teaching support) pinpointed inconsistency as one of the main challenges with digital badge use: “quality is going to vary greatly from subject/unit to subject/unit or programme to programme around how the badge is used and that inconsistency is going to undermine its value as well for the student”. 0% 10% 20% 30% 40% 50% 60% Australasian Journal of Educational Technology, 2021, 37(3). 111 Quality assurance and accreditation issues, which encompassed lack of formal recognition and lack of regulation, were identified by 10 of the 14 interviewees as a significant challenge to digital badge use. The following comments are characteristic of statements made by most interviewees: If it’s globalised, universally accepted, then it could go across all sorts of institutions, including private companies but it has to be globally accepted which will mean that you can pretty much on your phone carry your qualification. However, that has to be verified somehow and ratified. There has to be a formal way of doing it.” (William, university, academic development) The university has a responsibility to ensure that they are only awarded for a particular quality and who determines what that quality is? What role do the university faculty have, what role do the administration have in determining what that quality is, what role do students have? (Daniel, university, faculty) Aligned with the theme of quality assurance and accreditation issues was a further theme about the need for a coherent, integrated approach to digital badge adoption. For example, Kyle (university, academic development) spoke of the need for consistency: I think the consistency in programmes is an issue. I think the only way something like that would work, is if we had a consistent approach, which would allow us to have a clear structure and a way of working so that badging would make sense. Grace (university, faculty) identified the need for an institution-wide coherent approach: “the institution then must have some kind of a common understanding that they can put out to students saying that this is how we assist students, and this is the benefit”. One theme related to the challenges of digital badges that emerged from the qualitative data that was not present in the survey results was the need for a cultural or mindset shift within organisations for badges to be successfully implemented. Ian (university, leadership) talked about the need for “a really big shift in people’s mindset and I still don’t believe I have got there with that in terms of promoting what a digital badge can do.” Oliver (ITP, faculty) discussed the challenge of embedding digital badges into the culture of an organisation: “that’s a challenge. So, the understanding and the institution, I mean, they have to embrace it, and they have to embed it in the culture of the institution”. Current digital badge use Having gathered participants’ perceptions of the benefits and challenges associated with digital badges, they were then asked about the current use of digital badges within their institution. Of the responses, 53.5% (60/112) indicated that their institutions were using digital badges or intended to do so in the future, and 18% (20/112) answered that their institutions were not using badges. A further 16% (18/112) were unaware of whether their organisation was using badges or not or whether they planned to do so in the future. The remaining 12.5% (14/112) answered “other”. Survey participants were also asked to identify the digital badge system being used within their institution. Of the responses, 43% (46/107) used Moodle, 23.5 % (25/107) did not know, 15% (16/107) chose “other” and 10% (11/107) used Mahara. The remaining 8.5% (9/107) was made up of Blackboard, Canvas, in- house systems and Totara. Responses in the “other” category identified a range of other platforms, including iQualify, edX edge, Credly, BadgeOS and PeerWise. Respondents to the survey were then asked to indicate whether they had first-hand experience of digital badges. Of those who answered the question, 46% (52/114) of respondents answered “yes” and 54% (62/114) answered “no”. Those who answered “yes” were then asked how they had used badges with learners (choosing all that applied). The most common responses were to encourage participation from learners, to represent learning and to motivate learners (see Figure 3). Notably, 27% (12 respondents) indicated they had not used digital badges after previously answering “yes” to the first-hand experience Australasian Journal of Educational Technology, 2021, 37(3). 112 question. Rather than using badges with learners, these respondents had experienced earning badges as part of their own professional development. Figure 3. Use of digital badges with learners All 14 interviewees had first-hand experience of badge use. When asked about what they understood about badges, the most prominent theme that emerged was digital badges were signifiers of competency, learning or achievement. Comments such as “They are a form of a credential … my experience with them has been around how they have been used primarily for a means of acknowledging that somebody has achieved at a particular level” (Ian, university, leadership), “it’s a representation of an accomplishment or a skill, or an achievement” (Oliver, ITP, faculty) and “it is about showing that … they [learners] were able to perform to a certain level of competency for the next part of their journey” (Ari, university, faculty) were common. When asked about the level of badge use, 85% (44/52) of survey respondents across 13 institutions (who answered “yes” to having first-hand experience of badges) indicated that most badge use was occurring within individual subject or units (i.e., micro level) and relatively little activity was occurring across whole qualifications or at an institution-wide level (see Figure 4). The subject or unit leader (the person with overall responsibility for a subject or unit) was the most common person who decided what badges were used, followed by faculty in collaboration with colleagues, and the faculty member on their own. Qualitative findings also indicated predominant use was occurring at the micro (i.e., subject or unit) level, as the following comment indicates: We use digital badges in a course [subject/unit] … to indicate particular progress for us as faculty but also for the student to say that they have met the criteria and that they are ready to go onto the next stage. (Daniel, university, faculty) Institution-wide initiatives were mentioned but less often. A main reason for the lack of institution-wide implementation was identified by Oliver (ITP, faculty), who pointed out that “for it to work, I mean, the whole institution needs a structure around digital badges. Otherwise we’re just kind of shouting down a big hole”. 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Australasian Journal of Educational Technology, 2021, 37(3). 113 Those who chose the “other” category in the survey (see Figure 4) included comments such as “our staff development online courses [subject or unit] use them” (Respondent 50) and “for completion of small PD [professional development] activities” (Respondent 77). Using digital badges to recognise ongoing professional learning and development also emerged from the interviews. The following is indicative of comments made by the interview participants: We’ve attempted to use digital badges here under an initiative just last year … where we use Moodle as a learning management system and as a badging option for staff only. We offer opportunities to learn short course [subject/unit] pieces of information which can then be badged. (Thomas, ITP, leadership) Figure 4. Levels of digital badge use within institutions Future use intentions Survey participants who answered “no” to the first-hand experience question (62/114) were also asked about their intentions to use digital badges in the future (see Figure 5). Of the respondents who answered this question, at least 50% agreed (i.e., somewhat agreed, agreed or strongly agreed) with all statements. More specifically, results indicate that 83% of respondents agreed that the ease of use of digital badges would influence their willingness to use them in the future, and 81% agreed that the ease with which badges are set up would influence their willingness to use them in the future. Furthermore, 75% agreed they were interested to learn more about digital badges, and 73% agreed they needed to know more about digital badges. Only 59% agreed with the statement that they planned to talk with colleagues about digital badges. 0% 10% 20% 30% 40% 50% 60% Individual unit of study Learning activities in a unit of study Other Across qualification Institution-wide Australasian Journal of Educational Technology, 2021, 37(3). 114 Figure 5. Intention to use digital badges in the future Related to the need to learn more and know more about digital badges, the majority (12 out of 14) of interview participants highlighted the importance of a clear vision for introducing digital badges, as crucial to their successful implementation and use. In addition, staff support was also identified as critical to successful integration of digital badges. Gabriella (university, teaching development) identified the need for a clear vision: “I think you need to have a clear vision and a good partnership inside of the institution; and have a really good team as well, and have good people who can really implement that vision.” Emily (ITP, academic development) spoke about the need to involve staff in any initiative; one way of doing this was through a community of learning: If you think about any kind of innovation … you have a smaller group who trials or prototypes, pilots, you learn from them, before you roll out. … Then you can go, okay… these were the lessons learnt and you diffuse that across your school or across a community of learning. Discussion The results of this study indicate that digital badges were perceived to have a range of valuable attributes that can be categorised into three main areas. The three identified themes encompass digital badges as motivational support (e.g., as a motivational aid for learners, encouraging participation in learning); recognition of learning and achievement (e.g., display of achievement, as digital evidence of learning, as recognition of informal learning, and the ability of digital badges to be easily shared); and support for learning (e.g., support for assessment and as an aid for tracking progress for both learners and faculty). The main value of digital badges identified in this project reflects the main functions of digital badges identified in the literature review by Facey-Shaw et al. (2018). Furthermore, the findings presented here provide further empirical support for research and literature that highlights the potential of badges to impact learner motivation (Abramovich et al., 2013), evidence (formal and informal) learning (Ahn et al. 2014), support learning (Marshall, 2018), encourage participation (Chou & He, 2016) and share achievement over social networks (Casilli & Hickey, 2016). The finding that digital badges can be used to help learners regulate their own learning is important as the ability to self-regulate learning is recognised as a critical skillset (Duffy & Azevedo, 2015), and research related to digital badge use to support self-regulatory competencies is limited (Cucchiara et al., 2014). 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% I plan to talk with colleagues about digital badges Technical difficulties associated with digital badge setup and use will deter me from using badges in… I need to know more about digital badges before I use them I am interested to learn more about digital badges The ease with which digital badges are set up will influence my willingness to use them in the future The ease of use of digital badges will influence my willingness to use them in the future strongly disagree disagree somewhat disagree neutral somewhat agree agree strongly agree N/A Australasian Journal of Educational Technology, 2021, 37(3). 115 Badges were also considered to have some notable challenges that included staff lack of knowledge and training, inconsistent use, as well as the lack of formal recognition and regulation at multiple levels (e.g., department, institutional, government). The need for a coherent, integrated approach to digital badge adoption, which is likely to require an organisational cultural shift or change of mindset, emerged as an important finding from phase 2 of the study and reflects other research that has identified that difficulties experienced when introducing digital badges tended to be more social and cultural than technical (Mewburn et al., 2014). Significant challenges identified here such as inconsistent use of digital badges, the need for an integrated approach to implementation and the need for formal recognition and regulation processes beyond the individual subject or unit will require fundamental changes to existing institutional strategies and systems as has been noted previously (Lockley et al., 2016). Furthermore, the results have implications for actors beyond tertiary institutions, including New Zealand government agencies as well as companies who create the digital platforms on which digital badges and micro-credentials are stored and used. Similar to recent findings concerning micro-credential use in Australian universities (Selvaratnam & Sankey, 2019), the use of digital badges in New Zealand is still in the development stage and is not considered mature as usage is variable both within and between institutions. Moving from predominantly micro-level use of badges (which was found to be currently the case) to institutional and national level (macro-level) use will involve complex decisions and changes to educational systems and processes. Decision-makers need to be mindful that such changes are likely to require changes to organisational culture to ensure that introduced badging systems or micro-credentials succeed. Some of these changes are already underway; for example, the New Zealand Qualifications Authority (n.d.), responsible for ensuring qualifications are valued and credible, both nationally and internationally, has recently formally recognised micro-credentials within the qualification framework. Furthermore, the Tertiary Education Commission, who has governmental responsibility for the New Zealand tertiary education sector, has recently made funding available to tertiary education organisations “that can deliver well-designed micro-credentials to meet the needs of learners, communities, and industries, and support government priorities” (Tertiary Education Commission, 2019, ¶ 2). Staff development initiatives will need to be aware of the ease with which digital badges are established as well as the ease of ongoing use as the results of this study suggest that such concerns will impact future users’ willingness to adopt digital badges. In line with the reported findings, existing research highlights that as digital badge activity grows there will be a need for more organised approaches to staff training (Ellis et al., 2016). The finding that higher education staff are aware of potential drawbacks with using digital badges is an important consideration and highlights that badges need to be an integral part of the learning experience to ensure they are not simply viewed as tokens by staff or learners (Abramovich, 2016). In addition to identifying notable benefits and challenges, the findings also showed that over half of the respondents indicated that their institutions were already using digital badges or planned to use them in the future. Results also revealed that most badge use was occurring at subject or unit level (i.e., micro level) and those directly involved in these subject or units (i.e., subject or unit leaders and faculty) were responsible for making decisions about what badges were used and the criteria for awarding them. This finding is not unexpected given that implementation of badging systems can raise difficult questions about recognition, worth and trust (Willis et al., 2016). Conclusion and limitations Heeding the call for more empirical research (Carey & Stefaniak, 2018), the findings of this investigation represent an important contribution towards identifying and understanding digital badge use within public higher education in New Zealand. They offer valuable insights into current digital badge use and the extent of that use, as well as the perceived value and potential drawbacks of digital badges from staff who are using them or may do so in the future. The results highlight the importance of ensuring staff responsible for awarding digital badges understand their value and credibility and that they are involved in decisions about how and why badges are used beyond individual units of study. The main limitation with this research was the relatively low survey response rate when compared with the tertiary education workforce, which has over 30,000 employees, of whom 40% are classified as academic positions (Ministry of Education, 2017). Therefore, the results cannot be considered fully representative of the sector. Additionally, students’ experiences and perceptions of digital badges are missing from the Australasian Journal of Educational Technology, 2021, 37(3). 116 discussion. Subsequent phases of the overall research project will investigate the experiences of students with digital badges using a similar methodology to that reported here (i.e., national survey followed by individual interviews). The findings from this investigation add to the limited empirical research on digital badges and their use in the higher education sector particularly within the Australasian context. In doing so, they offer those with responsibility for designing and implementing digital badging systems valuable insights from which to develop and improve practice. References Abramovich, S. (2016). Understanding digital badges in higher education through assessment. On the Horizon, 24(1), 126–131. https://doi.org/10.1108/OTH-08-2015-0044 Abramovich, S., Schunn, C., & Higashi, R. (2013). Are badges useful in education? it depends upon the type of badge and expertise of learner. Educational Technology Research & Development, 61(2), 217–232. https://doi.org/10.1007/s11423-013-9289-2 Abramovich, S., & Wardrip, P. S. (2016). The impact of badges on motivation to learn. In L. Y. Muilenburg & Z. L. Berge (Eds.), Digital badges in education: Trends, issues, and cases (pp. 53–61). Routledge. Ahn, J., Pellicone, A., & Butler, B. S. (2014). Open badges for education: What are the implications at the intersection of open systems and badging? Research in Learning Technology, 22. https://doi.org/10.3402/rlt.v22.23563 Bazeley. (2018). Integrating analyses in mixed methods research. Sage. https://doi.org/10.4135/9781526417190 Berge, Z. L., & Muilenburg, L. Y. (2016). In the eye of the beholder: Value of digital badges. In L. Y. Muilenburg & Z. L. Berge (Eds.), Digital badges in education: Trends, issues, and cases (pp. 102– 108). Routledge. Carey, K. L., & Stefaniak, J. E. (2018). An exploration of the utility of digital badging in higher education settings. Educational Technology Research & Development, 66(5), 1211–1229. https://doi.org/10.1007/s11423-018-9602-1 Casilli, C., & Hickey, D. (2016). Transcending conventional credentialing and assessment paradigms with information-rich digital badges. The Information Society, 32(2), 117–129. https://doi.org/10.1080/01972243.2016.1130500 Chakroun, B., & Keevy, J. (2018). Digital credentialing: Implications for the recognition of learning across borders. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000264428 Chou, C. C., & He, S.-J. (2016). The effectiveness of digital badges on student online contributions. Journal of Educational Computing Research, 1–25. https://doi.org/10.1177/0735633116649374 Creswell, J. W. (2014). Research design international student edition: Qualitative, quantitative and mixed methods approaches (4th ed.). Sage. Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Sage. Cucchiara, S., Giglio, A., Persico, D., & Raffaghelli, J. E. (2014). Supporting self-regulated learning through digital badges: A case study. In Y. Cao, T. Väljataga, J. K. T. Tang, H. Leung, & M. Laanpere (Eds.), New Horizons in Web Based Learning: International Workshops SPeL, PRASAE, IWMPL, OBIE, and KMEL, FET (Vol. 8699, pp. 133–142). Springer. https://doi.org/10.1007/978-3- 319-13296-9_15 Devedžić, V., & Jovanović, J. (2015). Developing open badges: A comprehensive approach. Educational Technology Research & Development, 63(4), 603–620. https://doi.org/10.1007/s11423-015-9388-3 Duffy, M. C., & Azevedo, R. (2015). Motivation matters: Interactions between achievement goals and agent scaffolding for self-regulated learning within an intelligent tutoring system. Computers in Human Behavior, 52, 338–348. https://doi.org/10.1016/j.chb.2015.05.041 Ellis, L. E., Nunn, S. G., & Avella, J. T. (2016). Digital badges and micro-credentials: Historical overview, motivational aspects, issues, and challenges. In D. Ifenthaler, N. Bellin-Mularski, & D.-K. Mah (Eds.), Foundation of digital badges and micro-credentials: Demonstrating and recognizing knowledge and competencies (pp. 3–21). Springer. https://doi.org/10.1007/978-3-319-15425-1_1 European Commission. (2017). Strengthening European identity through education and culture. https://ec.europa.eu/commission/sites/beta-political/files/communication-strengthening-european- identity-education-culture_en.pdf https://doi.org/10.1108/OTH-08-2015-0044 https://doi.org/10.1007/s11423-013-9289-2 https://doi.org/10.3402/rlt.v22.23563 https://doi.org/10.4135/9781526417190 https://doi.org/10.1007/s11423-018-9602-1 https://doi.org/10.1080/01972243.2016.1130500 https://unesdoc.unesco.org/ark:/48223/pf0000264428 https://doi.org/10.1177/0735633116649374 https://doi.org/10.1007/978-3-319-13296-9_15 https://doi.org/10.1007/978-3-319-13296-9_15 https://doi.org/10.1007/s11423-015-9388-3 https://doi.org/10.1016/j.chb.2015.05.041 https://doi.org/10.1007/978-3-319-15425-1_1 https://ec.europa.eu/commission/sites/beta-political/files/communication-strengthening-european-identity-education-culture_en.pdf https://ec.europa.eu/commission/sites/beta-political/files/communication-strengthening-european-identity-education-culture_en.pdf Australasian Journal of Educational Technology, 2021, 37(3). 117 Facey-Shaw, L., Specht, M., van Rosmalen, P., Borner, D., & Bartley-Bryan, J. (2018). Educational functions and design of badge systems: A conceptual literature review. IEEE Transactions on Learning Technologies, 11(4), 536–544. https://doi.org/10.1109/TLT.2017.2773508 Fields, E. (2015). Making visible new learning: Professional development with open digital badge pathways. Partnership: The Canadian Journal of Library & Information Practice & Research, 10(1), 1–10. https://journal.lib.uoguelph.ca/index.php/perj/ Finkelstein, J., Knight, E., & Manning, S. (2013). The potential and value of using digital badges for adult learners (Final Report). American Institutes for Research. https://lincs.ed.gov/publications/pdf/AIR_Digital_Badge_Report_508.pdf Gibson, D., Ostashewski, N., Flintoff, K., Grant, S., & Knight, E. (2013). Digital badges in education. Education and Information Technologies, 20(2), 403–410. https://doi.org/10.1007/s10639-013-9291-7 Grant, S. L. (2016). History and context of open digital badges. In L. Y. Muilenburg & Z. L. Berge (Eds.), Digital badges in education: Trends, issues, and cases (pp. 3–11). Routledge. Groningen Declaration. (n.d.). Why Groningen declaration. https://www.groningendeclaration.org/ Hakulinen, L., Auvinen, T., & Korhonen, A. (2015). The effect of achievement badges on students' behavior: An empirical study in a university-level computer science course. International Journal of Emerging Technologies in Learning, 10(1), 18–29 https://doi.org/10.3991/ijet.v10i1.4221 Kehoe, A., & Goudzwaard, M. (2015). E-portfolios, badges, and the whole digital self: How evidence- based learning pedagogies and technologies can support integrative learning and identity development. Theory Into Practice, 54(4), 343–351. https://doi.org/10.1080/00405841.2015.1077628 Keevy, J., & Chakroun, B. (2015). Level-setting and recognition of learning outcomes: The use of level descriptors in the twenty-first century. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000242887 Lockley, A., Derryberry, A., & West, D. (2016). Drivers, affordances and challenges of digital badges. In D. Ifenthaler, N. Bellin-Mularski, & D.-K. Mah (Eds.), Foundation of digital badges and micro- credentials: Demonstrating and recognizing knowledge and competencies (pp. 55–70). Springer. https://doi.org/10.1007/978-3-319-15425-1_4 Marshall, S. (2018). Shaping the university of the future: Using technology to catalyse change in university learning and teaching. Springer. https://doi.org/10.1007%2F978-981-10-7620-6 Mewburn, I., Freund, K., & Rutherford, E. (2014). Badge trouble: Piloting open badges at the Australian National University. In B. Hegarty, J. McDonald, & S.-K. Loke (Eds.), Rhetoric and Reality: Critical Perspectives on Educational Technology: Proceedings of the 31st Annual ASCILITE Conference (pp. 643–648). https://www.ascilite.org/conferences/dunedin2014/files/concisepapers/233-Mewburn.pdf Ministry of Education. (2017). Profile and trends: Tertiary education sector and student support 2016. https://www.educationcounts.govt.nz/__data/assets/pdf_file/0003/183891/2016-pt-sector-supportb.pdf New Zealand Qualifications Authority. (n.d.). Recognising micro-credentials in New Zealand. https://www.nzqa.govt.nz/about-us/consultations-and-reviews/recognising-micro-credentials/#m-c- system O'Byrne, W. I., Schenke, K., Willis III, J. E., & Hickey, D. T. (2015). Digital badges: Recognizing, assessing, and motivating learners in and out of school contexts. Journal of Adolescent & Adult Literacy, 58(6), 451–454. https://doi.org/10.1002/jaal.381 Oliver, B. (2019). Making micro-credentials work for learners, employers and providers. Deakin University. https://dteach.deakin.edu.au/2019/08/02/microcredentials/ Peck, K., Bowen, K., Rimland, E., & Oberdick, J. (2016). Badging as micro-credentialing in formal education and informal education. In L. Y. Muilenburg & Z. L. Berge (Eds.), Digital badges in education: Trends, issues, and cases (pp. 82–92). Routledge. Reid, A. J., Paster, D., & Abramovich, S. (2015). Digital badges in undergraduate composition courses: Effects on intrinsic motivation. Journal of Computers in Education, 2(4), 377–398. https://doi.org/10.1007/s40692-015-0042-1 Roy, S., & Clark, D. (2019). Digital badges, do they live up to the hype? British Journal of Educational Technology, 50(5), 2619–2636. https://doi.org/10.1111/bjet.12709 Selvaratnam, R., & Sankey, M. (2019). Micro-credentialing as a sustainable way forward for universities in Australia: Perceptions of the landscape [White paper]. Australasian Council on Open, Distance and e-Learning. https://www.acode.edu.au/pluginfile.php/6495/mod_resource/content/1/ACODE White Paper SankeySelvaratnam.pdf Tertiary Education Commission. (2019). Micro-credentials funding approval. https://www.tec.govt.nz/funding/funding-and-performance/investment/plan-guidance/micro- credentials/ https://doi.org/10.1109/TLT.2017.2773508 https://journal.lib.uoguelph.ca/index.php/perj/ https://lincs.ed.gov/publications/pdf/AIR_Digital_Badge_Report_508.pdf https://doi.org/10.1007/s10639-013-9291-7 https://www.groningendeclaration.org/ https://doi.org/10.3991/ijet.v10i1.4221 https://doi.org/10.1080/00405841.2015.1077628 https://unesdoc.unesco.org/ark:/48223/pf0000242887 https://doi.org/10.1007/978-3-319-15425-1_4 https://doi.org/10.1007%2F978-981-10-7620-6 https://www.ascilite.org/conferences/dunedin2014/files/concisepapers/233-Mewburn.pdf https://www.educationcounts.govt.nz/__data/assets/pdf_file/0003/183891/2016-pt-sector-supportb.pdf https://www.nzqa.govt.nz/about-us/consultations-and-reviews/recognising-micro-credentials/#m-c-system https://www.nzqa.govt.nz/about-us/consultations-and-reviews/recognising-micro-credentials/#m-c-system https://doi.org/10.1002/jaal.381 https://dteach.deakin.edu.au/2019/08/02/microcredentials/ https://doi.org/10.1007/s40692-015-0042-1 https://doi.org/10.1111/bjet.12709 https://www.acode.edu.au/pluginfile.php/6495/mod_resource/content/1/ACODE%20White%20Paper%20SankeySelvaratnam.pdf https://www.acode.edu.au/pluginfile.php/6495/mod_resource/content/1/ACODE%20White%20Paper%20SankeySelvaratnam.pdf https://www.tec.govt.nz/funding/funding-and-performance/investment/plan-guidance/micro-credentials/ https://www.tec.govt.nz/funding/funding-and-performance/investment/plan-guidance/micro-credentials/ Australasian Journal of Educational Technology, 2021, 37(3). 118 Willis, J. E., Flintoff, K., & Mcgraw, B. (2016). A philosophy of open digital badges. In D. Ifanthaler, N. Bellan-Mularski, & D.-K. Mah (Eds.), Foundation of digital badges and micro-credentials: Demonstrating and recognizing knowledge and competencies (pp. 23–40). Springer. https://doi.org/10.1007/978-3-319-15425-1_2 Corresponding author: Maggie Hartnett, m.hartnett@massey.ac.nz Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Please cite as: Hartnett, M. (2021). How and why are digital badges being used in higher education in New Zealand? Australasian Journal of Educational Technology, 37(3), 104–118. https://doi.org/10.14742/ajet.6098 https://doi.org/10.1007/978-3-319-15425-1_2 mailto:m.hartnett@massey.ac.nz https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.6098 Introduction Methodology Data collection Participants Findings Survey demographics Benefits of digital badge use Challenges of digital badge use Current digital badge use Future use intentions Discussion Conclusion and limitations References