Irish Journal of Technology Enhanced Learning (2021) Volume 6, Issue 1 194 Participant Engagement with Eportfolio on the PACT Digital Badge: An Explanatory Sequential Mixed- Methods Analysis Marie O’ Neill*1 Ken McCarthy2 1CCT College Dublin 2Waterford Institute of Technology Abstract This paper explores participants’ perceptions of eportfolio on the National Forum for the Enhancement of Teaching and Learning’s PACT digital badge in which eportfolio is a central component of the teaching and learning experience. The PACT badge was launched in 2018 by the National Forum to promote engagement with their National Professional Development Framework for All Those Who Teach in Higher Education (National Forum, 2016). A small but growing body of studies indicates that the pairing of eportfolio with digital badges increases participant engagement with course content, facilitates the open evidencing of skill acquisition as well the transfer of knowledge beyond the life span of the course (Kehoe and Goudzwaard, 2015; Ambrose and Anthony 2016; Nilson and Dewiyanti, 2019). In January 2021, an explanatory, sequential mixed methods approach comprising a survey and focus group was employed to obtain and analyse feedback from graduates of three iterations of the PACT digital badge in relation to their perceptions of the eportfolio component. Data indicated a range of benefits including enhanced digital literacy, an enhanced ability to reflect, a more engaging learning experience, more robust evidencing of professional development and increased engagement with the National Professional Development Framework for all Those Who Teach in Higher Education. Seventy-Eight percent of respondents stated that eportfolio was an important part of the PACT course. A key recommendation of the study is the additional front loading of support materials in relation to eportfolio engagement in Week 0 of the course in response to feedback in which PACT participants indicated that eportfolio could be technically challenging and time consuming. The findings of this exploratory study could be tested at larger scale across a wider range of National Forum digital badges that incorporate eportfolio. 1.Introduction This paper explores participants’ perceptions of eportfolio on the National Forum for the Enhancement of Teaching and Learning’s PACT digital badge in which the construction of an eportfolio is a central component of the teaching and learning experience. The PACT digital badge promotes engagement with the National Forum for the Enhancement of Teaching National Professional Development Framework for All Those Who Teach in Higher Education (National Irish Journal of Technology Enhanced Learning (2021) 195 Forum, 2016). A small but growing body of studies indicates that the pairing of eportfolio and digital badges into the design and delivery of courses increases participant engagement with course content, the open evidencing of skill acquisition as well the transfer of knowledge beyond the life span of the course. This study seeks to augment the body of literature on eportfolio use in the Irish context with specific reference to digital badges and the National Professional Development Framework for all Those Who Teach in Higher Education (National Forum, 2016). 1.1 Background The National Forum for the Enhancement of Teaching and Learning’s PACT short course was launched in 2018, with the term ‘pact’ symbolising professionals’ personal pact or commitment to professional development. The badge promotes engagement with the National Forum’s National Professional Development Framework for all Those Who Teach in Higher Education (National Forum, 2016). The Framework recommends the pursuit of professional development according to four domains and five typologies. See fig 1 & 2. Fig. 1 The Domains of the National Professional Development Framework for All Those Who Teach in Higher Education (National Forum, 2016, p.5) Irish Journal of Technology Enhanced Learning (2021) 196 Fig 2: Typologies of the National Professional Development Framework for All Those Who Teach in Higher Education. The PACT course is structured, non-accredited, non-formal (National Forum, 2016, p.2) The PACT course which culminates in the awarding of a digital badge, is a part of a suite of open courses developed by Ireland’s National Forum which recognises engagement through a “peer triad digital badge ecosystem” (Donnelly and Maguire, 2020, p.1). Donnelly and Maguire (2020, p.6) observe that: The open courses with digital badges were developed for a national rather than an institutional context and for recognition rather than formal accreditation, and is an important initiative for Irish higher education. The courses are short (25 hours of learner effort), accessible (allowing participant choice across three modes: face-to-face/blended; self-study; online) and provide recognition to participants who have meet agreed criteria by awarding a National Forum digital badge. This national endorsement supports staff employment mobility by giving credibility to these badges. Referencing the work of Hickey et al (2015); Donnelly and Maguire (2020) state that stronger connections between digital badges, eportfolios and credit for prior learning, seem like promising directions for increasing the perceived value of badges. Participants of the PACT badge must complete an eportfolio to evidence learning on the programme. The construction of an eportfolio during the course is also designed to promote and scaffold participants’ on-going engagement with the National Forum’s Professional Development Framework for All Those Who Teach beyond the lifespan of the course. Irish Journal of Technology Enhanced Learning (2021) 197 This paper seeks to explore the perceptions of participants on the PACT digital badge course in relation to the eportfolio component. The study will explore whether the eportfolio component of PACT enhanced the learning experience and supported on-going engagement with the National Forum for the Enhancement of Teaching and Learning digital badge upon cessation of the course. 2. Literature Review Digital badges are electronic symbols used to document performance and achievement. Open badges are intended to provide additional information via metadata in order for viewers of badges to verify issuer details, evaluation criteria, and evidence such as the actual work product used to earn the badge (Parker, 2015). Parker also describes digital badges as an effective form of alternative assessment which support collaboration and engagement; enables students to reflect upon their progress as well as evidence skill attainment to employers. Farrell (2018, p.154) describes eportfolio as fulfilling a range of functions as “a tool or technology, a practice, a pedagogical model, an assessment method and a framework for learning.” Farrell, (2020, para. 45) in her exploration of the history of eportfolio identifies two key areas in relation to recent developments in higher and further education. The first relates to the impact of eportfolio on student learning and the second on “pairing eportfolios with emerging edtech such as digital badges, block chains and massive open online courses.” Eynon and Gambino (2017) identify the pairing of eportfolio and digital badges as an important area for future eportfolio research. This study also explores the convergence of digital badges and eportfolio and its impact on the learning experience and professional development of participants. Four principle themes have emerged in the small but growing body of literature on the pairing of eportfolio and digital badges. Articles explore the role of digital badges and eportfolio in underpinning the open evidence of learning; in promoting student motivation and persistence; in synthesising and transferring learning beyond the campus environment and the pedagogical challenge of integrating eportfolios into digital badges. Ambrose and Anthony (2016, para. 24) indicates that the integration of eportfolio with digital badges makes “learning visible” and “brings “MOOC learners alive with a face and a story that shows proof of performance competency.” A 2019 case study by Nilson and Dewiyanti highlights how digital badge metadata can be used to evidence achievement which can in turn be “uploaded and displayed in the students’ PeddlePad eportfolios demonstrating the students..core skills.” Weller (2018) explores the role of blockchain with “the aim to provide an individual, portable record of educational achievement.” A 2015 study by Kehoe and Goudzwaard explores how the pairing of digital badges with eportfolios promotes integrative learning and identity development so that students graduate from “making simple connections among ideas and experiences to synthesizing and transferring learning to new, complex situations within and beyond the campus.” This is a particularly powerful finding in the context of this study which also seeks to explore whether use of eportfolio on the PACT badge promotes engagement with the National Forum’s National Professional Development Framework beyond the life cycle of the PACT badge. Irish Journal of Technology Enhanced Learning (2021) 198 Scully, O Leary and Brown in their 2018 review on learning portfolios describe some of the challenges of pairing digital badges with eportfolio. They highlight how the use of digital badges act as an incentive representing a form of extrinsic motivation to learn “but one of the ultimate goals of learning portfolio practice is to foster intrinsic motivation to learn.” (p.18) Buchem 2016 cited in Scully, O’Leary and Brown’s article, (2018, p.349) cautions against shifting the focus from learning to “badge-collecting”. Gibson et al (2016) also cited in Scully, O’Leary and Brown, (2018, p.349) advocates for the use of “Inward facing” badges which are less formal and “can be awarded by peers or even by the learner themselves, and do not necessarily adhere to formal open badging infrastructures. It may be that this format is a better match for the pedagogy of the learning portfolio.” Cheng and Chau (2013) recommend that learners have dual goal orientation whereby they focus on both the process and product elements of their eportfolios which result in higher levels of persistence and reflective activity. The challenge of using eportfolio to openly evidence learning in a digital badge context versus engaging with eportfolio as a more introspective and personal exercise without outside incentive or pressure is also explored in this study. Gibson’s description of the peer awarding of badges as a solution to this challenge is particularly interesting given the fact that National Forum open courses are awarded by peer triads. 3. Methodology Exploring the perceptions of PACT participants on eportfolio was deemed by the authors to be best served by a mixed methods research approach. Creswell and Plano Clark, (2011) outline the numerous benefits of a mixed methods approach comprising qualitative and quantitative research methodologies, chief of which is the synergistic utilisation of data which will provide more in- depth insights than the employment of one research method alone. This study employs an explanatory sequential design with an initial quantitative phase followed by a qualitative phase in which the qualitative data is used to elucidate quantitative findings (Creswell and Plano Clark, 2018). Mixed methods studies provide a breath of perspective and understanding with data grounded in individuals' lived experiences. They also increase the validity and rigor of findings through the triangulation of data. In a mixed methods sequential analysis, meta-inference is obtained through a narrative reporting process in which both qualitative and quantitative findings are explored concurrently on a theme-by theme basis (Guetterman, Fetters and Creswell, 2015). The concurrent exploration of quantitative and qualitative data according to emergent themes in the data is executed in this study. Bryant and Chittum (2013) in their review of eportfolio research address the limitations of research focusing on student perceptions of eportfolio highlighting how students do not always prefer the instructional methods that result in the greatest learning gains. They state that that it is difficult to know whether those that had negative perceptions of an eportfolio’s impact on their learning felt that way because the tool itself was flawed, or because there were flaws in its implementation.” (p. 194). This insight was also considered when analyzing data emanating from the study. Irish Journal of Technology Enhanced Learning (2021) 199 3.1 Data Collection and Analysis One hundred and six individuals completed the first three iterations of the National Forum’s digital badge between 2018 and 2020. To ascertain the perceptions of PACT participants of the portfolio component of the badge, an anonymous survey using Survey Monkey comprising closed and open questions was distributed to all graduates of the three iterations eliciting a response rate of 30%. The consent process was integrated into the survey tool, incorporating background information to the study and information on the use and storage of data. A focus group was subsequently conducted comprising a range of higher and further education personnel who were recruited using purposive sampling to ensure a cross section of insight and opinion. A consent form was disseminated to focus group participants prior to the execution of the focus group outlining the nature of the research and guaranteeing the anonymisation of data. Focus group data was coded as part of a grounded theory approach to identify concepts and construct theory (Corbin and Strauss, 2008). Grounded theory is inductive in its approach, attempting to identify and understand how and why people behave the way they do. Ethics approval was obtained for the paper from the Research Committees of CCT College Dublin and Waterford Institute of Technology respectively prior to the commencement of the study which took place between January and May of 2021. 4. Findings 4.1 Demographics Feedback to the survey emanated from a wide range of higher and further education institutions. The two largest groups were the university and the institute of technology sectors representing 27% and 33% of responses respectively. See Fig.3. Irish Journal of Technology Enhanced Learning (2021) 200 Fig. 3 What type of academic institution are you working in? Fifty-Six percent of respondents were lecturers followed by librarians at 16%. Responses were also obtained from educational developers, academic managers and learning technologists. Fig. 4 Which of the following describes your role in education? 27% 33% 15% 9% 6% 3% 6% 0% 5% 10% 15% 20% 25% 30% 35% 56% 3% 3% 3% 0% 19% 16% Lecturer Academic Manager Educational Developer Learning Technologist IT Professional Librarian Other (please specify) 0% 10% 20% 30% 40% 50% 60% Irish Journal of Technology Enhanced Learning (2021) 201 Focus group members comprised professionals from the university, private higher education and ETBI sectors in a range of roles. The authors were not successful in recruiting participants to the focus group from the IOT sector however. Please see Table 1 for details pertaining to the composition of the group. Focus group member Role 1. Student Success Manager and ICT Lecturer 2. Quality Assurance Officer 3. Senior Academic Manager 4. Lecturer in Library Science 5. Assistant Librarian 6. Instructional Designer 7. Librarian 8. Professional Learning and Development Coordinator 9. Senior Academic Librarian Table 1: Focus group members The largest proportion of survey responses were from the science and engineering disciplines followed by social science, technology, business and technology. See Fig. 5. Fig. 5 If you are a lecturer, please indicate which discipline you teach in? 0% 14% 5% 10% 0% 0% 10% 0% 5% 0% 10% 5% 0% 10% 14% 19% 0% 5% 10% 15% 20% 25% Irish Journal of Technology Enhanced Learning (2021) 202 4.2 Themes This study employs an explanatory sequential design which comprises an initial quantitative phase followed by a qualitative phase. Qualitative data is used to elucidate quantitative findings (Creswell and Plano Clark, 2018). Meta-inference has been obtained through a narrative reporting process in which both qualitative and quantitative findings are explored concurrently on a side by side and theme-by theme basis (Guetterman, Fetters and Creswell, 2015). The following themes emerged from this methodological approach. 4.2.1 The underutilisation of eportfolio in higher education in Ireland Farrell, (2018) discusses how “eportfolio has not emerged at scale in Irish Higher Education” (p.154), adding that this trend is “out of alignment with the ubiquitous nature of eportfolio in higher education worldwide”. Farrell describes drivers of eportfolio use globally as concerted and includes government policy and flourishing communities of practice. Farrell (2018) also describes a lack of “explicit government policy” (p.154) in relation to the adoption of eportfolio in higher education in Ireland, “compared to the government policy drivers in the UK, U.S., Australia and New Zealand” (p.156). She describes how a rapidly expanding and energetic community of practice in Ireland, along with the work of National Forum for the Enhancement of Teaching and Learning is progressing eportfolio adoption in higher education in Ireland. Survey findings in this study reinforce Farrell’s article. Fifty-four respondents had not used eportfolio prior to starting the PACT digital badge. Of those who had used eportfolio, 59% had been using it for a year or less. Only 15% stated that they had been using eportfolio for more than three years. Additionally, 55% stated that their institution did not provide support for eportfolio use. Twenty-Four percent stated that they did receive support for training in relation to eportfolio; 18% in relation to hosting and 12% in relation to technical support. See Fig 6. Fig. 6 Range of supports provided by respondents’ institutions in relation to eportfolio 24% 18% 12% 55% Training Hosting Technical Support My institution does not provide support for e- portfolio 0% 10% 20% 30% 40% 50% 60% Irish Journal of Technology Enhanced Learning (2021) 203 Only 24% of PACT participants used an existing eportfolio during the course. The remainder created a new one. WordPress was the most popular platform for those creating a new eportfolio with 45% of respondents choosing it compared to 21% of participants who chose to use the National Forum’s Mahara instance. See Fig. 7. In response to the question what influenced your choice of eportfolio platform (participants were requested to select one or more reasons). Seventy-five percent of respondents cited ease of use as the reason for their choice, followed by cost at 25%. Twenty-Two percent stated that they availed of the National Forum’s Mahara instance; 3% because the platform they chose integrated with an institutional license and 3% because of institutional support. Fig. 7 Please indicate which eportfolio platform you used during the PACT course? The majority of focus group members had no prior experience of eportfolio prior to the start of the course. Focus group members 1 and 2 were required to complete eportfolios as part of Masters programmes that they undertook. Having completed PACT, they regretted not maintaining these earlier portfolios however. The majority of focus group members stated that eportfolio adoption within their institutions for teaching and learning and professional development purposes remained ad-hoc. Focus group member 9 suggested that a strategic approach to eportfolio adoption was required nationally and institutionally. Focus group member 2 stated that institutions involved in blended learning have been proactively using eportfolio for teaching and learning purposes for many years. Its use within these institutions has also grown during the Covid-19 pandemic. She was less aware of eportfolio being used as extensively by staff for professional development purposes. 21% 45% 0% 9% 24% The National Forum's Mahara platform Wordpress Pepplepad Weebly Other (please specify) 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Irish Journal of Technology Enhanced Learning (2021) 204 4.2.2 The complexity of Mahara as an eportfolio platform The National Forum’s PACT digital badge provides access to course participants to the Mahara platform in perpetuity. The Mahara instance drew mixed responses however with just 22% stating that they managed it from the outset. Fifty-Six percent of respondents stated that they found the Mahara platform confusing initially but worked their way through it, whilst 11% found it difficult to use throughout. Finally, 11% of respondents stated that they did not like the platform and had abandoned it. Open ended survey responses in relation to Mahara reinforced these findings. Responses included the following statements: “I had to invest a huge amount of time in learning how to use it. I didn't find it user friendly….I ended up with many folders/subfolders and a portfolio that was poorly laid out.” “Lots of potential in this platform especially re sharing content with other people. I think that it needs more scaffolding (even two hours of instruction) prior to the start of the badge.” “PACT was my third attempt at Mahara across different institutions. It had not gotten any better so I stuck with WordPress” “ I use Mahara in my teaching, as it's the platform my institution supports. I hate the interface, it is very inefficient” “It was difficult to navigate” The majority of focus group members also found Mahara difficult to use. Focus group member 3 stated that she reviewed Mahara in relation to possible institution wide adoption but feedback from staff and students, indicated that they found it clunky and difficult to use. Google Sites was chosen instead. She had also found Mahara difficult to use as a student on the PACT badge to the point that it occasionally undermined her intention to complete the course. Focus group member 6 stated that she didn’t like the aesthetics of the Mahara platform and found the customisation aspect burdensome. By contrast she has maintained a Wordpress eportfolio which she established on an alternative programme of study. She also used this eportfolio during PACT. 4.2.3 Time management issues A dominant theme in the survey and focus group findings was the difficulty of finding time to engage with eportfolio during the PACT digital badge course. Both survey respondents and focus group members felt that the time spent on eportfolio detracted from their engagement with course content on the PACT digital badge. Open ended survey comments also indicated a frustration with the time spent on constructing an eportfolio. One participant described how ’a lot of time’ was spent ‘trying to learn how to use Wordpress,’ which was “disappointing as I had invested so much time and energy into the course with the intention of having my eportfolio for professional Irish Journal of Technology Enhanced Learning (2021) 205 development ready to go.” Another described how “Eportfolio set up can be very time consuming as a novice. It can detract from reflection on the actual content” All focus group members found engagement with eportfolio on the PACT course time-consuming. Focus group member 8 stated that if you were a novice who gave serious consideration to your professional persona and identity, the amount of time it took to configure the portfolio appropriately, compromised engagement with course content. Focus group member 2 also felt that the time that she devoted to the eportfolio aspect of the programme eroded her ability to engage with course content. She undertook the course “to reflect” and “not to take a digital badge in eportfolio”. Focus group member 2 used a Wordpress eportfolio during the PACT digital badge and didn’t maintain it upon completion of the course. She now uses a spreadsheet to maintain a list of her professional development activities in which she maps entries to the National Professional Development Framework. Focus group member 2 stated that it would have been helpful to have ready-made templates for those who do did not have the time to be too creative with the eportfolio aspect of the badge. These templates would support the PACT participant to create an appropriate eportfolio for the course and for on-going professional development in an efficient and time effective manner. 4.2.4 Sustainability Sixty percent of respondents agreed or strongly agreed that they planned to use eportfolio going forward to engage with the National Professional Development Framework for All Those Who Teach in Higher Education. Survey data revealed the reasons why the remaining PACT participants stated that they wouldn’t. Reasons included lack of time and technical expertise. See Fig.8. Open ended survey comments included: “More scaffolding of support at the start and end of the course to ensure effective engagement during the badge but also beyond the lifetime of the badge. Perhaps where a NF course has an eportfolio dimension students should have to complete a badge on eportfolio first.” Focus group member 9 highlighted the growing importance of LinkedIn for capturing and evidencing professional development activity as a potential threat to eportfolio. She uses OneNote for her eportfolio but has struggled to maintain the platform due to time constraints. Focus group member 2 now tries to reflect on themes as opposed to individual conferences and events which she finds a more time effective approach. Stating that she “loves a deadline”, she wonders if the National Forum would prompt PACT participants to “write a reflection on a theme every quarter”. This would support the sustainability of eportfolios beyond the life span of the course. Irish Journal of Technology Enhanced Learning (2021) 206 Focus group member 3 found the process of maintaining a portfolio and aligning entries to the National Forum’s Professional Development Framework helpful in terms of reflecting on her professional development but she has struggled to maintain the eportfolio over the last year. Focus group member 6 stated that the COVID-19 pandemic eroded her motivation to maintain her eportfolio with the extra demands placed on staff professionally during this period. Focus group member 8 stated that the pandemic made staff much more strategic about what they maintain and what they don’t. She felt that she struggled with the eportfolio aspect of the digital badge and that she wanted to get more comfortable with eportfolio before she rolls it out to staff. She added that the pandemic has made “people more experimental but also more focused”. Fig 8. If you plan not to continue with eportfolio after the PACT course to engage with the National Forum's National Professional Development Framework, please indicate why. 4.2.5 Earlier provision of support for eportfolio on the PACT digital badge In response to open ended questions on how interaction with the eportfolio component of the course could be improved, survey respondents and focus group members stated that the front loading of training prior to the commencement of the badge was critical. Open-ended comments included: “More instruction on eportfolio before the course starts. “ 47% 6% 18% 6% 0% 24% Lack of time Lack of training Lack of technical expertise Lack of institutional support Lack of confidence Other (please specify) 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Irish Journal of Technology Enhanced Learning (2021) 207 “Spend one session doing a workshop showing how to attach material to it. Demonstrate how to structure it to get a good layout.” “Demonstrate it in a simple way.. a Noddy's guide!” “If exercises were submitted on Mahara that would have encouraged me to use it” “Demos of pros and cons of different portfolio software” “Need to survey PACT participants prior to course and to offer an eportfolio basics course to participants who have no prior experience of eportfolios.” “I think a skeleton of components that can make up a functional eportfolio would be helpful. It's not easy to start from scratch without knowledge of the platform.” Focus group members also advocated the front loading of additional training and resources in relation to eportfolio use on the PACT digital badge. In comparison to survey respondents, several focus group members highlighted the educational and support role of eportfolio communities of practice in Ireland. Focus group member 6 observed that there was a much higher prevalence of literature and events pertaining to the embedding of eportfolio into teaching and learning practice. Much of this insight and guidance is disseminated by experts on social media platforms such as Twitter. All focus group members were aware of eportfolio Ireland as an example. Focus group member 4 highlighted the importance of institutional champions to increase engagement with and support the adoption of eportfolio. Focus group member 1 described how the eportfolio aspect of the PACT digital badge commences in week two or three advocating that engagement with eportfolio should commence in “Week 0”. Focus group member 3 facilitated an iteration of the PACT badge and promoted Google sites due to its ease of use, producing an accompanying video which was provided to participants prior to the commencement of the programme. Consequently, complaints about the eportfolio component of the course were less prevalent on this iteration of the badge. 4.2.6 Eportfolio promotes engagement with course content and with the National Framework for All Those Who Teach in Higher Education Seventy-Two percent of survey respondents said that they better understood the National Forum’s National Professional Development for All Those Who Teach in Higher Education as a result of the eportfolio component of the course. 63% of survey respondents agreed or strongly agreed that eportfolio helped them to capture and document course work for the peer triad discussions. Less positive scores were elicited for the role Irish Journal of Technology Enhanced Learning (2021) 208 of eportfolio in promoting collaboration. Forty-four percent agreed or strongly agreed that eportfolio facilitated collaboration with other learners. Fifty-four percent agreed or strongly agreed that eportfolio facilitated the submission of assessment tasks. Forty-four percent agreed or strongly agreed that eportfolio underpinned feedback from facilitators. Thirty-six percent of participants strongly agreed and 42% agreed that eportfolio was an important part of the PACT digital badge. Overall, 62% of respondents stated that the learning experience on the PACT course was enhanced by use of eportfolio. Eight-one percent agreed or strongly agreed that eportfolio promoted reflection. Open ended survey comments included: “It was a good way to work your way through the Framework. I would have liked to see more sharing via the portfolio. There was very little interaction on the actual Mahara portfolio platform. Most of the interaction was through the Moodle platform for the course. Although I am sure that people did go in and view each others’ portfolios.” “The eportfolio offers huge potential to engage with the National Framework for the Professional Development of All Those Who Teach in Higher Education” “Eportfolio on the PACT digital badge provided a structure to unpick the complexity of the Framework and the many different elements” “It gave a focus and a space to consider CPD activities under the Framework, and without it I would have been much less inclined to recognise the influence of informal, collaborative activities.” “Useful to have a resource that you can continue to update if you wish - that's why I keep my on my own Squarespace site” “There is more reflection when using an eportfolio so it gets you thinking what you want to do career wise.” Open ended survey data indicated that PACT digital badge reinvigorated portfolios that participants had prior to the start of the course. Comments include: “Yes, I added more content and grew in confidence re eportfolio use “Yes, as I revisited my eportfolio and after reflection on the NPDF I made changes.” “Just in relation to the categorisation of posts. My writing of content continues as normal on a monthly basis (mostly).” “It did result in my adding to my previously completed eportfolio” Irish Journal of Technology Enhanced Learning (2021) 209 “Yes. It caused me to look back at it and reflect on how I was using it. I had not been updating it as often as I felt I should have been.” “ Yes, as I revisited my eportfolio and after reflection on the NPDF I made changes. “Yes, I added more content and grew in confidence re eportfolio use” “I already had a portfolio, nothing fancy but it supported my learning. The framework allowed me to categorise my learnings and update all my posts over the previous three years.” The majority of focus group members felt that eportfolio promoted reflection. Focus group member 6 felt that the triads were a more effective mechanism for reflection however. Focus group member 5 felt that having a pre-existing and established eportfolio prior to the commencement of the PACT badge made the learning process far more beneficial and rewarding. 4.2.7Additional skills provided by eportfolio on the PACT badge Survey and focus group participants highlighted the acquisition of a wide range of additional skills and opportunities as a result of the eportfolio component of the PACT digital badge. See table 2 for survey and focus group feedback on additional skills acquired. Statement provided in the survey Percentage of survey respondents that agreed or strongly agreed Focus group feedback Eportfolio enhanced my digital skills 69% Focus group member 6: “as an instructional designer, I was interested to learn new eportfolio platform” Eportfolio enhanced my critical thinking skills 66% Focus group member 3 “I do like the eportfolio. It made me think in a structured way. I want to empower learners and faculty.” Eportfolio enhanced my ability to reflect 81% Focus group member 2: “eportfolio provides a space for reflection” Focus group member 8: “eportfolio provides space for structured reflection” Irish Journal of Technology Enhanced Learning (2021) 210 Eportfolio gave me a stronger sense of professional identity 81% Eportfolio enhanced communication with peers 63% Focus group member 9 – “I enjoyed networking and connection with other people particularly librarians” Eportfolio helps me to evidence learning outcomes 75% Focus group member 4. “eportfolio provides a space to consolidate all your PD and learning” Eportfolio helps me to capture and document course work 81% Eportfolio gives me a platform to demonstrate my career/PD development 75% Focus group member 5 “eportfolio provides a record of everything that you have done” The learning experience on the PACT course has enhanced my use of eportfolio as part of my professional development activities 62% Focus group member 1 “eportfolio provides a push to engage with PD” Eportfolio is an important part of the PACT course 78% Focus group member 3. Eportfolio helps you to put the “National Professional Development Framework into practice.” Table 2: Additional Skills from the eportfolio element of the PACT Badge 4.2.8 The inclusion of eportfolio on the PACT badge promotes engagement with eportfolio in teaching and learning Sixty-nine percent of survey respondents agreed that their knowledge of eportfolio had increased as a result of their participation on the PACT course whilst 60% agreed or strongly agreed that their confidence with eportfolio had increased as a result of being on the PACT course. Sixty percent of survey respondents agreed or strongly agreed that they planned to use eportfolio going Irish Journal of Technology Enhanced Learning (2021) 211 forward to engage with the National Professional Development Framework for All Those Who Teach in Higher Education. Sixty-two percent agreed or strongly agreed that the learning experience has enhanced use of eportfolio in their own teaching practice. Sixty-five percent of survey respondents agreed or strongly agreed that they planned to take knowledge they had acquired as a result of participating on the PACT digital badge back to their institutions. Seventy- eight percent of survey respondents stated that eportfolio should remain a central component of the PACT digital badge. Eighty-four percent of survey participants felt that integration of eportfolio into professional practice courses is an effective way to promote engagement with eportfolio by staff in education. Open comments include: “Eportfolios were a useful tool to demonstrate professional development in PACT” “Using an eportfolio was a great way to complete this course however I think using Mahara has taken away from the course. You should look for a better alternative. Even a simple blogging tool would be better I think” “The eportfolio was a fantastic repository for my PD and helped me develop an action plan. “ “While it was a challenge and the process was uncomfortable at times, and finding the time was not easy, it overall was a very positive experience. It really helped to take the experience and learning to a deeper level, and to have a real sense of personal ownership of the process.” “I think eportfolios are a useful tool for this type of learning” “It has helped me document and reflect on my learning.” “I enjoyed looking at the eportfolios of the other participants to see how they approached it” “Provide a separate 'how to' open course on creating an eportfolio for those who teach” Focus group members 4 and 9 are working with the National Forum and the Library Association of Ireland to produce a version of the PACT digital badge to promote uptake amongst library personnel at all levels sector wide. Focus group member 3 stated that she encourages all new programmes within her institution to incorporate eportfolios. Having the facilitator badge for PACT made her more aware of promoting eportfolio generally. 5. Conclusions An explanatory sequential mixed methods design has served a number of functions in this paper providing a more in-depth analysis of themes from the survey data as well as the triangulation of findings for additional rigor and validity. In this study, focus group findings and the literature Irish Journal of Technology Enhanced Learning (2021) 212 review corroborate and expand upon a number of key concepts emanating from survey data supporting the generalisation and transferability of findings which is particularly important given the relatively small survey sample size. This study has found that the pairing of eportfolio with the PACT digital badge enhances the learning experience, promotes reflection and additional skill acquisition as well as on-going engagement with the National Professional Development Framework for All Those Who Teach in Higher Education. These findings could be tested on a larger scale across National Forum open courses that have an eportfolio component. Bryant and Chittum (2015) in their review of the eportfolio literature indicates that learners do not always appreciate a teaching approach or tool even if it is beneficial for them. In the context of this paper, this was not the case. This may be in part because of the nature of the learner on the PACT course which is a higher or further education professional. Seventy-Eight percent of participants deemed the eportfolio component of the PACT badge as central to the delivery of the course and the learning experience. Data emanating from both the survey and focus group did not indicate a tension between the extrinsic motivation of a digital badge and the more intrinsic, introspective and reflective elements of an eportfolio. 6. Recommendations A key finding of this study is that eportfolio on the PACT badge enhances the learning experience, develops other key skills such as reflection and digital literacy and promotes engagement with the National Framework for Professional Development for All Those Who Teach in Higher Education. Survey and focus group participants indicated that the time and technical knowledge required for platforms such as Mahara in the context of a short six-week course are key challenges however. A key recommendation of this study pertains to the additional front loading and scaffolding of information and training on eportfolio prior to the commencement of the programme in Week 0. Another key recommendation is the provision of eportfolio templates or components as per focus group feedback to underpin more effective and efficient eportfolio engagement on the PACT course. Whilst 60% of PACT participants indicated that they would maintain engagement with their eportfolio upon cessation of the course, consideration could also be given to the provision of additional support to scaffold post-PACT eportfolio engagement to increase this percentage. This model of eportfolio support could be extrapolated to all digital badges offering eportfolio. Irish Journal of Technology Enhanced Learning (2021) 213 References Ambrose, AG, Anthony, E and Clark, GC. (2016). Digital badging in the MOOC space. Educause Review, Nov. Available at: https://er.educause.edu/articles/2016/11/digital-badging-in-the-mooc- space. Accessed 30th March, 2021. Bryant, L and Chittum, J. 2013. Eportfolio effectiveness: An (ill-fated) search for empirical support. International Journal of ePortfolio, 3(2): 189–198. Buchem, I. (2016). Digital Badges at (Parts of) Digital Portfolios: Design Patterns for Educational and Personal Learning Practice. In: D. Ifenthaler, N. Bellin-Mularski & D. Mah (Eds.) Foundation of Digital Badges and Micro-Credentials. (pp.343-368). Switzerland: Springer Bryman A. (2006). Integrating quantitative and qualitative research: how is it done? Qualitative Research, 6, 97–113. doi: 10.1177/1468794106058877. Corbin, J., & Strauss, A. (1996). Analytic ordering for theoretical purposes. Qualitative Inquiry, 2(2), 139-150. Creswell JW, Plano Clark VL., (2011). Designing and conducting mixed methods research. 2nd ed. Thousand Oaks, CA: Sage. Creswell JW, Plano Clark VL., (2018). Designing and conducting mixed methods research. 3rd ed. Los Angeles: Sage Publications. Donnelly, R and Maguire, T. (2020), Building Digital Capacity for Higher Education Teachers: Recognising Professional Development Through a National Peer Triad Digital Badge Ecosystem, European Journal of open Distance and E-Learning, 23 (2), January 2021. Available at: https://doi.org/10.2478/eurodl-2020-0007. Accessed 30th March, 2021. Eynon, B and Gambino, L. (2017). High impact eportfolio practice. Sterling, VA: Stylus. Farrell, O., (2020). From Portafoglio to Eportfolio: The Evolution of Portfolio in Higher Education. Journal of Interactive Media in Education, 2020(1), p.19. DOI: http://doi.org/10.5334/jime.574. Accessed 30th March, 2021. Farrell, O., (2018). Failure to Launch: Failure to Launch: The Unfulfilled Promise of Eportfolios in Irish higher education: An Opinion Piece, DBS Business Review. Vol. 2., pp. 155-163. Available at: https://www.dbsbusinessreview.ie/index.php/journal/article/view/30. Accessed 20th March, 2021. https://jime.open.ac.uk/articles/10.5334/jime.574/main-text-B3 https://er.educause.edu/articles/2016/11/digital-badging-in-the-mooc-space https://er.educause.edu/articles/2016/11/digital-badging-in-the-mooc-space https://doi.org/10.2478/eurodl-2020-0007 https://jime.open.ac.uk/articles/10.5334/jime.574/main-text-B31 http://doi.org/10.5334/jime.574 https://www.dbsbusinessreview.ie/index.php/journal/article/view/30 Irish Journal of Technology Enhanced Learning (2021) 214 Gibson, D., Coleman, K. & Irving, L. (2016). Learning Journeys in Higher Education: Designing Digital Pathways Badges for Learning, Motivation and Assessment. In: D. Ifenthaler, N. Bellin- Mularski & D. Mah (Eds.) Grush, M. (2015). Showcasing the Co-Curricular: ePortfolios and Digital Badges. Retrieved from: https://campustechnology.com/articles/2015/01/27/showcasing-the-co-curricular- witheportfolios-and-digital-badges.aspx. Accessed 30th March, 2021. Guetterman TC, Fetters MD, Creswell JW (2015). Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Ann Fam Med 2015 Nov;13(6):554-561 Hickey, D.; Willis, J.; Quick, J. (2015). Where Badges Work Better. Open Badges in HE. Available at: https://sites.google.com/site/openbadgesinhighereducation/where-badges-work- better. Accessed November 1, 2021. Kehoe, A., & Goudzwaard, M. (2015). ePortfolios, Badges, and the Whole Digital Self: How Evidence-Based Learning Pedagogies and Technologies Can Support Integrative Learning and Identity Development. Theory into Practice, 54(4), 343–351. Lloyd, M. (2015). Pairing Eportfolios with Badges to Document Informal Learning. Retrieved from: https://campustechnology.com/articles/2015/07/29/pairing-eportfolios-with-badges- todocument-informal-learning.aspx. Accessed 30th March, 2021 National Forum for the Enhancement of Teaching and Learning in Higher Education, (2016) "National Professional Development Framework for all Staff Who Teach in Higher Education," in teachingandlearning.ie, Accessed April 1, 2021, https://www.teachingandlearning.ie/publication/national-professional-development- framework-for-all-staff-who-teach-in-higher-education/. Nilson, C & Dewiyanti,, S (n.d.) Supporting nursing students’ self-directed learning and competency achievements through the Murdoch University, Discipline of Nursing PebblePad Digital Badge Program Available at: https://resources.pebblepad.co.uk/hubfs/PebblePad%20- %20Murdoch%20University%20-%20Digital%20Badges%20-%20Case%20Study.pdf. Accessed 4th April, 2021. Parker, H. E. (2015). Digital badges to assess bloom’s affective domain. The National Teaching & Learning Forum, 24(4), 9–11. DOI :10.1002/ntlf.30031. Accessed 29th March, 2020 Rhodes, T. (Ed.). (2010). Assessing outcomes and improving achievement: Tips and tools for using rubrics. Washington, DC: Association of American Colleges and Universities cited in Kehoe, A., & Goudzwaard, M. (2015). ePortfolios, Badges, and the Whole Digital Self: How Evidence-Based Learning Pedagogies and Technologies Can Support Integrative Learning and Identity Development. Theory into Practice, 54(4), 343–351. https://campustechnology.com/articles/2015/01/27/showcasing-the-co-curricular-%20witheportfolios-and-digital-badges.aspx https://campustechnology.com/articles/2015/01/27/showcasing-the-co-curricular-%20witheportfolios-and-digital-badges.aspx https://sites.google.com/site/openbadgesinhighereducation/where-badges-work-better https://sites.google.com/site/openbadgesinhighereducation/where-badges-work-better https://campustechnology.com/articles/2015/07/29/pairing-e-portfolios-with-badges-todocument-informal-learning.aspx https://campustechnology.com/articles/2015/07/29/pairing-e-portfolios-with-badges-todocument-informal-learning.aspx https://www.teachingandlearning.ie/publication/national-professional-development-framework-for-all-staff-who-teach-in-higher-education/ https://www.teachingandlearning.ie/publication/national-professional-development-framework-for-all-staff-who-teach-in-higher-education/ https://resources.pebblepad.co.uk/hubfs/PebblePad%20-%20Murdoch%20University%20-%20Digital%20Badges%20-%20Case%20Study.pdf https://resources.pebblepad.co.uk/hubfs/PebblePad%20-%20Murdoch%20University%20-%20Digital%20Badges%20-%20Case%20Study.pdf Irish Journal of Technology Enhanced Learning (2021) 215 Scully, D., O’Leary, M. & Brown, M. (2018). The Learning Portfolio in Higher Education: A Game of Snakes and Ladders. Dublin: Dublin City University, Centre for Assessment Research, Policy & Practice in Education (CARPE) and National Institute for Digital Learning (NIDL). Available at: https://www.dcu.ie/sites/default/files/carpe/docs/eportfolioreport.pdf. Accessed 29th March, 2021. Weller, M. (2018). Twenty years of Edtech. Educause Review, 53(4): 34–48. Available at https://www.educause.edu/ https://www.dcu.ie/sites/default/files/carpe/docs/eportfolioreport.pdf https://www.educause.edu/ Participant Engagement with Eportfolio on the PACT Digital Badge: An Explanatory Sequential Mixed-Methods Analysis Marie O’ Neill*1 Ken McCarthy2 1CCT College Dublin 2Waterford Institute of Technology Abstract This paper explores participants’ perceptions of eportfolio on the National Forum for the Enhancement of Teaching and Learning’s PACT digital badge in which eportfolio is a central component of the teaching and learning experience. The PACT badge was ... 1.Introduction 1.1 Background 2. Literature Review 3. Methodology 3.1 Data Collection and Analysis 4. Findings 4.1 Demographics 5. Conclusions 6. Recommendations References