Irish Journal of Technology Enhanced Learning (2021) Volume 6, Issue 1 174 Looping Everyone into the Conversation: The Use of Eportfolio as a Multistakeholder Feedback Tool Roisin Lyons*1 Ciara Lynch2 Eoghan McConalogue3 1 University of Limerick 2 Dublin City University 3Dublin City University Abstract At its most fundamental, entrepreneurship involves discovering, evaluating, and exploiting opportunities to create future goods and services (Shane and Venkataraman, 2000). Acknowledging that characteristics, emotions, cognitive biases, and past experiences influence entrepreneurial activity and decision making (Shepherd and Patzelt, 2017; De Winnaar and Scholtz, 2019), many propose that active reflection is an important facet of the entrepreneurial curricula (Nabi et al., 2016; Santos et al., 2016). For the entrepreneurial student, learning to make rationale decisions is paramount, and this is vitally linked to their ability to reflect and be self-aware. This study recounts the use of the eportfolio within a new large class (over 600 students) module in enterprise education. The module utilised an eportfolio reflective assignment to allow students express their feelings and knowledge about a series of attended entrepreneurial events and guest speaker seminars. In this paper, we present a novel insight into the efficacy of this curricular approach. External stakeholders who acted as mentors and speakers were asked to review a number of these student portfolios, and provide their thoughts on the assignments themselves, and the e-portfolio construct more generally. As such, this study highlights the multiple feedback loops of reflection which can be obtained from the eportfolio when used as a carefully considered pedagogical tool. It also highlights the integral role that industry stakeholders play in the enterprise curriculum. 1. Introduction This article discusses the value of the eportfolio as an assessment tool to depict student learning and development in the enterprise education space. It also highlights the usefulness of the tool at showcasing student innovation and reflection to external stakeholders, and as a feedback mechanism to engage in wider curricular review. Within a large undergraduate enterprise module, students attended an external entrepreneurship festival, and participated in a social Irish Journal of Technology Enhanced Learning (2021) 175 innovation hackathon event as part of their course, both of which were highly attended by industry representatives and entrepreneurs (as mentors, judges, speakers and attendees). Following these events, students were asked to create academic reflections about these experiences on their personal Loop Reflect¹ eportfolio pages, tying together a range of visual, literary, and personal aspects. A selection of excerpts and student blogs posts emanating from this experience were shown to six innovation industry experts, who were asked to comment on the experience and their impression of the eportfolio submissions. The innovation experts noted that the reflective approach provides students additional space to connect with the entrepreneurs and their relevant academic reading. Furthermore, the experience provided additional food for thought for these stakeholders, thereby encouraging them along a reflective journey of their own. We argue that eportfolio generate multiple opportunities for layered and nuanced feedback, peer and stakeholder engagement, and functionality that extends far beyond the assessment domain. 2. Reflective Practice in the Enterprise Classroom Enterprise education helps to develop business-related soft skills and competencies in a wide range of students, not just those intent on starting a venture. Aimed at creating ‘entrepreneurs’, ‘entrepreneurial’ and ‘enterprising individuals’ the broadest consideration of this subject domain aims to develop a self-reliance and enterprising mind-set regardless of career path (Fayolle et al., 2006). Lackéus (2017) suggests that enterprise/entrepreneurial education allows a student to become more creative and innovative. While the subject is constructed to impart knowledge and develop skills, it is also intended to foster an ecosystem conducive to effective entrepreneurial growth. In their study, Charney and Libecap (2000) found entrepreneurship graduates are three times more likely to start their own business; three times more likely to be self-employed; have annual incomes 27% higher, own 62% more assets; and are more satisfied with their jobs. Entrepreneurial modules and courses within a university setting are predominantly taught via traditional business lectures blended with experiential activities and assignments. Educators often utilise novel teaching and assessment methods to allow students to enhance student development of innovation, ideation, and abstraction (Dong, Garbuio and Lovallo, 2016; Garbuio et al., 2018). Recent developments see courses which include themes such as effectuation, business model canvas, lean start-up and design thinking (Lackéus, 2015; Ramsgaard and Christensen, 2016); hackathons (Clinton and Lyons, 2020); and mentorship (Hägg and Politis, 2017). At the curriculum level, most Irish entrepreneurship courses use the creation of a business-plan as the major focus (Clinton and Lyons, 2020). Reflective practice is a useful pedagogical choice for the enterprise educator for a number of key reasons. Noting that there are fewer tangible benefits to enterprise education modules, it is essential that the changes in student confidence, intentionality and other factors are also seen and celebrated. We consider that the use of reflection, using a platform such as the eportfolio, is well-suited to capture both the performance aspects related to an enterprise module, and the cognitive and affective developments of the individual student also. Irish Journal of Technology Enhanced Learning (2021) 176 For a student to become a self-regulated learner, the development of the capacity to make judgements and decisions is aided by their ability to reflect and be self-aware (Harms, 2015; Hägg, 2021). The applied and experiential activities which are synonymous with entrepreneurial education give students exposure to pseudo real-world experiences of entrepreneurship (Kirby, 2004; Neck and Greene, 2011). These simulated journeys can provide them with a safe space to learn about themselves as potential innovators and venture creators or risk takers. Reflection in this context allows the learner the freedom to evaluate and justify their own decision-making, which promotes metacognition (Shepherd et al., 2014). Classroom instruction about specific entrepreneurial knowledge (for example feasibility, financial theory, marketing, legal implications or funding) provides an occasion for learners to consider and benchmark their own skillset (or lack thereof). It may provide them with a basis for self- reflection and pause to evaluate their own entrepreneurial skills, attitudes and proclivities, noting their reservations and skill development plan in a reflective blog or planning exercise. Reflective practice in the enterprise classroom can also aid the student entrepreneur in their future decision making through the instruction of entrepreneurial frameworks and thinking models. In academic literature, the study of entrepreneurial cognition is fundamental to examine how and why the entrepreneur thinks, reflects and acts (Michell et al., 2002). The teaching of entrepreneurial frameworks and structures are considered to be elements of cognitive entrepreneurship education, with the premise that “by teaching our students the characteristics, patterns and processes of thinking related to entrepreneurship, it is possible to shape their thinking patterns accordingly” (Santos et al., 2016, p. 90). By implementing these as modes of assessing choices, the use of heuristics in the decision making of entrepreneurs could be reduced. Without better practices in knowledge sourcing, data collection and accountability, there may be more risk of poor decision making due to cognitive bias (De Winnaar and Scholtz, 2019). Thus, the fundamental act of reflective thinking and the awareness of one’s own decision making processes could have significant benefits for the enterprise student, in the short and longer term contexts. From the perspective of the educator the eportfolio and reflective process offers a novel opportunity to observe student development over time and to assist in real time (McManus, 2021). Most instances of the eportfolio and reflective process involve multiple student diary entries across over time (Hägg, 2021). Educators, therefore, have an opportunity to observe student reflection and development. This allows the educator to analyse student ability to both consume the practical experience and their ability to relate this experience with theory, as it occurs. As such, in many cases an educator can facilitate ‘Vygotsky scaffolding’ or simply a scaffold learning experience (Shvarts and Bakker, 2019). If an educator observes a student struggling to link theory and the practical learning, they can assist the student in developing this skill in real time, an opportunity that is not often available in traditional methods of business education. 3. Methodology This paper is contextualised within the third-level level education system in Ireland, focusing on EE at undergraduate level. The student sample group are from a large enterprise module of Irish Journal of Technology Enhanced Learning (2021) 177 over 600 first year students. This module itself exposes participants to multiple modes of learning including online learning, conference seminars, applied projects and reflective writing. Students develop their enterprise skills and knowledge via lectures, teamwork, online learning, applied project work, and industry speaker events. Students gain insight into the reality of the business world across a range of organisational types from corporate business to SME or family business, entrepreneurship to non-profit contexts. They investigate processes of bringing new ideas into action within these contexts, through the lens of enterprise and innovation. The module is designed to highlight the spectrum of business career options open to our students upon graduation. The mission of the module is to develop innovative and enterprising competencies in business students, develop their enterprise self-efficacy, and allow them to witness the importance and applicability of these competencies in all enterprise contexts. Figure 1: Student Portfolio Sample As part of the curriculum, students attended an external entrepreneurship festival, and participated in a social innovation hackathon event, both of which were highly attended by industry representatives and entrepreneurs (as mentors, judges, speakers, and attendees). Following these events, students were asked to create academic reflections about these Irish Journal of Technology Enhanced Learning (2021) 178 experiences on their personal Loop Reflect eportfolio page, tying together a range of visual, literary, and personal aspects (See Figure 1 of a student reflection including their discussion and addition of images and multimedia). To facilitate students in this process, the reflective blogs were templated with question boxes to guide student open responses. For example, “How do these experiences link to your previous reading or knowledge (Please refer to specific academic sources)”, and “What did you learn about the entrepreneurial process from this experience?”. Students were provided with workshops on reflective thinking and academic writing, a recommended reading list of just five academic articles to limit quality and search issues for the first-year group, and online support videos. These assignments were graded by the module staff as per the specifications of the marking scheme, and the students received their marks and feedback on their blogposts. Once the module had been completed, and to gain a new and fresh reaction to the student contributions, a selection of these student blogs posts were presented to six innovation industry experts, who were asked to comment on the experience and their impression of the eportfolio submissions. To assess the shareholders’ perception of the student learning experience, we organised 4 eportfolio into 8 different folders. Each folder held 4 eportfolio of differing grades, ranging from 43% to 82% (note: marks were not shown or released to the external reviewers). Each eportfolio was randomly chosen based on the grade that the student received and all confidential information, or personal information which affected student anonymity, was removed or redacted. Each eportfolio was assigned a number, for which the corresponding name of the student and link to their eportfolio were kept in a separate file, only accessible to the research team. Ethical permissions were provided to conduct the study within the college. The study was conducted one year after the students submitted work, and the data collection process took approximately 12 weeks in total. The industry experts were contacted by email to be a part of the study. Once they were in agreement, they were sent a folder containing the files as a shared Google file (view only), and asked to submit their responses via a Google form. These experts were asked to spend some time reading and reviewing the samples, then answer some questions about their impression of the quality of the submissions, their views on the curricular method itself, and the importance of industry in enterprise education. Once the study was completed, guest access to these files was revoked for the external reviewers. 4. Results and Discussion The panel was made up of a range of six experts from the innovation and business field, ranging from academia to consulting. The group collectively have an impressive level of experience and are a high-calibre expert panel for the study. Firstly, the group were asked to provide their reaction to the reflections they had read (See Table 1). All statements were positive, with members stating they were impressed or pleasantly surprised, or were interested the aspects which were salient to the students. One member noted that “It was very interesting to read about what the students thought and what resonated with them”, highlighting the value of such an exercise in connecting these industry leads to the student psyche. Irish Journal of Technology Enhanced Learning (2021) 179 Role What was your perception of the reflections you read? Professor & Innovation Consultant I was impressed with the ability of the students to articulate the impact the experiences had on them. Business Consultant I was pleasantly surprised with quality of these reflections particularly as these are first year students. Lots of openness and honesty on display on their before and after understanding of starting a business and entrepreneurship. Allowing the students to choose their events, tapped into their own interests which (after reading) and provided role models who also seem to have inspired many of the students. It was also refreshing to read about failure and how it is important to recognise failure as a learning opportunity on the entrepreneurial journey. Education and Health Innovation Manager Overall, the students appeared to be engaged and although some have gone to extra lengths to provide references and further research all seem to have input an effort into the work. The secondary questions appear to provide more useful self-reflection opportunities for the students. Economic Development Researcher It was very interesting to read about what the students thought and what resonated with them. For me it was clear that the experience had a strong impact on each of the four students. The students greatly appreciated the honesty among the entrepreneurs and stated various key learnings that they would seek to personally take on board going forward. Lecturer (Business) The reflections showed a varying degree of interaction with the reflective process. The differences displayed are identical to the differences displayed in my own cohort of students, using a similar method of reflection. Certain students have a greater ability to create connections with literature/prior knowledge and the events that they have attended Head of Programme & Engagement (Social Enterprise Initiative) I thought they were really interesting. It is clear that the students took a lot away from the events they attended. Table 1: Expert Perceptions of Student Reflections Next, reviewers were asked to provide their thoughts on the importance of industry to the enterprise student (See Table 2). Many referred to the value that students obtain in connecting with the real-world context of the entrepreneur, or a live business project, providing them with the opportunity to develop practical skills for the workplace. Three of the review panel also noted that it is advantageous for industry to connect with students also, to gain insight about this generation and their needs. Irish Journal of Technology Enhanced Learning (2021) 180 Role Why do you think it is important for Industry to get involved with enterprise students? Professor & Innovation Consultant It is important for students to be exposed to the start-up community to get the perspective of entrepreneurs who are implementing much of what they are learning in class. Business Consultant I think it is win-win for both with the university acting a bridge. The students benefit with doing work that is related to 'live' projects (and applying their learning) and industry (SMEs) benefit from being part of the continuous learning ecosystem and getting fresh perspectives from the next generation of leaders. Education and Health Innovation Manager It provides authentic learning experiences and is mutually beneficial. Students learn how to communicate and work within the boundaries of a firm and organisations keep in touch with the minds of a new generation of workers. Economic Development Researcher I think it is absolutely vital for industry and enterprise students to engage with each other - so that students can learn directly about what is actually involved in being an entrepreneur or developing an enterprise - and also so that industry can stay in tune with trends in the future labour and consumer markets. I think when students get to engage directly with industry, they can much more easily apply concepts, academic models, and theories - and also learn a great deal. Lecturer (Business) While theoretical learning is important, industry interaction with students enhances the practical learning experience. When the practical experience is introduced in the correct manner this can enhance student ability to appreciate the theoretical aspects of course content, something that can be challenging for some students. Head of Programme & Engagement (Social Enterprise Initiative) One of the biggest things is education. It appears that the students learned a lot in their classes but took a lot away from the personal experiences of the guest speakers. It is clear their insights have made an impact on the students. Table 2: Expert Opinion on Industry Engagement Lastly, reviewers were asked to reflect on the activity of attending the guest seminar and the eportfolio assessment itself as a pedagogical tool (See Table 3). All reviewers were in favour of the approach. Members stated the assessment type was ‘novel’, ‘valuable’, encourages more deep level thinking and in fact, may encourage more engagement in the enterprise network and events. The call to connect the experience with academic reading was commended as a way to bring the theory to life, and the guiding questions were also commented on as a good way to move students past rote descriptions of the event. Irish Journal of Technology Enhanced Learning (2021) 181 Role Do you think that this type of assessment is valuable for students? Why/ Why not? Professor & Innovation Consultant Yes. 1. They participate in a unique and engaging learning experience. 2. They are required to reflect on the experience which enhances and integrates the learning. Business Consultant I think this 'reflections' assessment exercise was very worthwhile and valuable for the students learning. While the students were required to attend online events/ watch recordings, they also had to stop and think and apply the learning to better understand the concept of entrepreneurship. The requirement to apply learning from the academic literature brings the subject matter to life. Education and Health Innovation Manager Yes. John Dewey mentions that we learn from our reflections on doing and I believe this is a good way to do this. The secondary questions guide the students away from mindlessly listing what someone has said and instead focus them on the application in their own personal context where there is more value. Economic Development Researcher Yes, I think it is very valuable as it: 1. Enables students to learn directly from peers / role models / real life examples of relatable business people 2. Encourages students to link theory / academic reading to practice / real life 3. Asks students to state how the new knowledge learned / experience will benefit them (impact). Lecturer (Business) It is invaluable. For the reasons given in the answer to the previous question, but also because this type of assessment gives students insights to challenges not always experienced until their first venture into full time employment. Such insight is beneficial not only to the student (industry knowledge prior to employment, for example) but also to a prospective employer (prospective employees have ready to use insights / speeds up training, for example). Head of Programme & Engagement (Social Enterprise Initiative) Absolutely - it gives the students an opportunity to reflect on what they have learned. This might then prompt them to register for another event, sign up to volunteer for a society or do more research into that specific area of interest. Table 3: Expert Opinion on Reflective Pedagogy 5. Conclusion Industry engagement in the form of guest speakers, industry judges and mentors are commonly used to inspire, motivate, and build self-efficacy in students (Bosma, Hessels, Schutjens et al., 2012). When used in enterprise education, they can help to create positive perceptions of entrepreneurship as a career (Nabi et al., 2016; Nabi et al., 2017). The entrepreneurial classroom becomes a controlled environment to engage in trial-and-error behaviours and experience the emotions inherent in low-risk failure (Shepherd, 2004). Armed with a greater awareness of the ‘bigger picture’, students can internalise and assimilate their entrepreneurial experiences, fostering an entrepreneurial mindset (Santos et al., 2016). We suggest the use of reflection (via reflective eportfolio blogs) is an effective way to develop the entrepreneurial Irish Journal of Technology Enhanced Learning (2021) 182 mindset in an enterprise education context. Moreover, we consider that this pedagogical approach is feasible and useful in both the large-class context, or small. In a smaller group, more time can be spent in facilitated classroom discussion about the industry examples, allowing students to learn from the reflective thinking and argumentation of their peers. In a larger context, much of this content can be moved to a blended format, with online videos about reflective practice and template examples. As a scaled model for larger classes, eportfolio- based assessment is readily accessible for module staff, negates the need for paper-based assignments, allows for multiple grader access in parallel, and allows students to showcase their creative, innovative, and multimedia competencies far more than a traditional document submission format. While vastly beneficial, industry involvement in courses such as enterprise education can pose a burden to staff and guests alike, increasing the administrative burden on all stakeholders. Thus, any involvement must be carefully considered to ensure industry experts feel involved, but not over-committed. Increasingly, institutions try to involve industry in the planning and creation of relevant modules and subjects to arm our next graduates. By integrating external voices in multiple dimensions of the educational process, more novel insight can be gained, and potentially more impact can be achieved than the traditional didactic guest seminar approach. We find that by drawing these experts into the final stages of an enterprise module, and by allowing industry guests to see the student output, they become more informed about student talent, student needs, and more aware of the pedagogical foundations at play within the module. Our research found the panel to have a deepened appreciation for the curricular methods of the module, which may lead to more informed future guest speakers, and a more robust link between industry and academia. Irish Journal of Technology Enhanced Learning (2021) 183 References Bosma, N., Hessels, J., Schutjens, V., Van Praag, M. & Verheul, I. (2012). Entrepreneurship and role models. Journal of Economic Psychology, 33(2), pp. 410-424, doi:10.1016/j.joep.2011.03.004. Charney, A. and Libecap, G.D. (2000). The Impact of Entrepreneurship Education: An Evaluation of the Berger Entrepreneurship Program at the University of Arizona, 1985- 1999. SSRN Electronic Journal. Clinton, C & Lyons, R (2020). Entrepreneurial and Career Intentions and Behaviour of Students attending Irish Higher Educational Institutes in Ireland: GUESSS Report 2019, International GUESSS Project, doi: 10.13140/RG.2.2.14849.48486 Dong, A., Garbuio, M. and Lovallo, D. (2016). Generative Sensing: A Design Perspective on the Microfoundations of Sensing Capabilities. California Management Review, 58(4), pp. 97– 117. De Winnaar, K., Scholtz, F., (2019). Entrepreneurial decision-making: new conceptual perspectives. Management Decision, 58(7), pp. 1283-1300. Fayolle, A., Gailly, B. and Lassas‐Clerc, N. (2006), Assessing the impact of entrepreneurship education programs: a new methodology, Journal of European Industrial Training, Vol. 30 No. 9, pp. 701‐20. Garbuio, M., Dong, A., Lin, N., Tschang, T., & Lovallo, D. (2018). Demystifying the genius of entrepreneurship: How design cognition can help create the next generation of entrepreneurs. Academy of Management Learning & Education, 17(1), pp. 41-61. Garbuio, M., & Lin, N. (2019, July). Abductive Reasoning in New Venture Ideas. In Academy of Management Proceedings (Vol. 2019, No. 1, p. 15663). Briarcliff Manor, NY 10510: Academy of Management. Hägg, G. (2021). The entrepreneurial diary: a reflective learning activity to enhance the judgmental abilities of student entrepreneurs. International Journal of Entrepreneurial Behaviour & Research, 27(5), pp.1142–1165. Hägg, G., and D. Politis. 2014. Learning through Reflection: The Role of Mentorship in Entrepreneurship Education. In Proceedings of the Entrepreneurship Summer University at ISCTE-IUL 2013, edited by A. Caetano, S. C. Santos, and S. F. Costa, 47–56. Lisbon: Instituto Universitário de Lisboa. about:blank Irish Journal of Technology Enhanced Learning (2021) 184 Harms, R. (2015). Self-regulated learning, team learning and project performance in entrepreneurship education: Learning in a lean startup environment. Technological Forecasting and Social Change, 100, pp. 21–28. Kirby, D.A. (2004). Entrepreneurship education: can business schools meet the challenge? Education and Training, 46(8/9), pp. 510-519. Lackéus, M. (2017). Can Entrepreneurial Education Escape Being Caught Between Marginal (‘the Devil’) and Irrelevant (‘the Deep Blue Sea’) Practices? In ECSB 3E Entrepreneurship Education Conference, pp. 10-12. McManus, R. (2021). Lessons from reflective journaling in undergraduate eportfolios. Irish Journal of Technology Enhanced Learning, 6(1), pp. 29-36. https://doi.org/10.22554/ijtel.v6i1.81 Mitchell, R.K., Busenitz, L., Lant, T., McDougall, P.P., Morse, E.A. and Smith, J.B. (2002). Toward a Theory of Entrepreneurial Cognition: Rethinking the People Side of Entrepreneurship Research. Entrepreneurship Theory and Practice, 27(2), pp. 93–104. Neck, H.M. & Greene, P.G. (2011). Entrepreneurship Education: Known Worlds and New Frontiers. Journal of Small Business Management, 49(1), pp. 55-70. Nabi, G., Liñán, F., Krueger, N., Fayolle, A. & Walmsley, A. (2017). The Impact of Entrepreneurship Education in Higher Education: A Systematic Review and Research Agenda. Academy of Management Learning & Education 16(2), pp. 277-299. Nabi, G., Walmsley, A., Liñán, F., Akhtar, I. & Neame. C. (2016). Does entrepreneurship education in the first year of higher education develop entrepreneurial intentions? The role of learning and inspiration. Studies in Higher Education, 43(3), pp. 1–16. Ramsgaard, M.B., Christensen, M.E., (2016). Interplay of entrepreneurial learning forms: a case study of experiential learning settings. Innovations in Education and Teaching International 55(1), pp. 1-10. Santos, F.J., Roomi, M.A., Liñán, F., (2016). About gender differences and the social environment in the development of entrepreneurial intentions. Journal of Small Business Management 54(1), pp. 49-66. Shane, S. & Venkataraman, S. (2000). The promise of entrepreneurship as a field of research. Academy of Management Review 25(1), pp. 217–226. Shepherd, D., Williams, T. & Patzelts, H. (2014). Thinking about entrepreneurial decision making: Review and Research Agenda. Journal of Management, 41(1), pp. 11-46. Shepherd, D. & Patzelt, H. (2017). Researching Entrepreneurial Decision making, Trailblazing in Entrepreneurship, Palgrave McMillian: Switzerland. https://doi.org/10.22554/ijtel.v6i1.81 Irish Journal of Technology Enhanced Learning (2021) 185 Shvarts, A. & Bakker, A. (2019). The early history of the scaffolding metaphor: Bernstein, Luria, Vygotsky, and before. Mind, Culture, and Activity, 26:1, pp. 4-23. Roisin Lyons*1 Ciara Lynch2 Eoghan McConalogue3 1 University of Limerick 2 Dublin City University 3Dublin City University 1. Introduction