Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 93 Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills Judy Currey1,2,4, Stephanie K. Sprogis1,3, Gabby Burdeu1, Julie Considine1,3,4, Joshua Allen1, Elizabeth Oldland1,4 judy.currey@deakin.edu.au; s.sprogis@deakin.edu.au; gabby.burdeu@deakin.edu.au; julie.considine@deakin.edu.au; josh.allen@deakin.edu.au; elizabeth.oldland@deakin.edu.au 1School of Nursing and Midwifery, Deakin University 2Deakin Learning Futures, Office of the Deputy Vice Chancellor (Education), Deakin University 3Centre for Quality and Patient Safety Research – Eastern Health Partnership 4Centre for Quality and Patient Safety Research, School of Nursing and Midwifery, Deakin University Abstract In tertiary education, generic professional skills should be developed along with discipline- specific knowledge and skills. Team-Based Learning (TBL), an active learning strategy, creates deep learning and enhanced student engagement; however, its effects on the development of generic learning outcomes are unknown. The aim of this study was to evaluate postgraduate specialty nursing students’ perspectives of how TBL impacts the acquisition of skills defined by the university’s eight Graduate Learning Outcomes (GLOs). A descriptive exploratory design was used in this study. Postgraduate nursing students in 2016-2017 at one university were invited to participate. Data were collected via demographic survey, a ranking tool, and written reflections. Data were analysed using descriptive statistics and content analysis. The response rate was 97.2 per cent (172/177). Participants were mostly females (n=152, 88.4%) aged 25–34 years (n=115, 66.9%). Student (n=156) rankings showed TBL contributed to the acquisition of critical thinking (n=90, 57.7%) and problem solving skills (n=56, 35.9%) the most. Students (n=144) made 2719 comments regarding how TBL led to the acquisition of GLOs in written reflections. Almost 98 per cent (n=2657) of all reflective comments were positive. All students mentioned at least one GLO positively due to TBL. Most positive reflections related to self-management (n=520, 19.6%) and communication (n=434, 16.3%). Postgraduate specialty nursing students perceived TBL classes contributed to the acquisition of their university’s GLOs, particularly critical thinking, problem solving, and self-management skills. The active learning strategy of TBL facilitates learning and engagement, and the attainment of essential professional attributes which are highly valued by employers. Keywords: Team-Based Learning, graduate attributes; professional skills, active learning, higher education, reflective practice, employability, postgraduate education mailto:judy.currey@deakin.edu.au mailto:s.sprogis@deakin.edu.au mailto:gabby.burdeu@deakin.edu.au mailto:julie.considine@deakin.edu.au mailto:josh.allen@deakin.edu.au mailto:elizabeth.oldland@deakin.edu.au Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 94 Introduction Over the last two decades, there has been an increasing focus on the role of universities to prepare graduates for employment. The definition of employment has broadened from Yorke’s original definition (2006) to now mean that students and graduates can discern, acquire, adapt and continually enhance the skills, understandings and personal attributes that make them more likely to find and create meaningful paid and unpaid work that benefits themselves, the workforce, the community and the economy (Oliver, 2015, p. 56). Academic leaders now specify that generic graduate learning outcomes must be translated into tangible learning and evidenced through assessments appropriate to the discipline. Generic graduate learning outcomes common to universities include the skills of: written and oral communication; critical and analytical thinking; information literacy; learning and working independently and collaboratively; and engaging ethically and inclusively with communities, cultures and nations (Oliver, 2011). For graduate health professionals, generic graduate learning outcomes such as well-developed team skills, reflective practice, self-management, and problem solving are critical for the development of essential behaviours known to improve the quality of healthcare (World Health Organisation [WHO], 2011). Critical care nurses provide care for the sickest, most vulnerable patients in acute care hospitals and their educational preparation and provision of evidence-based care have a direct impact on mortality, complications, and other patient outcomes (Henneman et al., 2010). Graduates of postgraduate critical care nursing courses are expected to meet professional practice standards (Nursing and Midwifery Board of Australia [NMBA], 2016), and the performance standards of specialty professional bodies (Australian College of Critical Care Nurses, 2015; College of Emergency Nursing Australasia, 2013) in addition to university graduate learning outcomes. To accelerate postgraduate critical care nursing students’ attainment of graduate learning outcomes, and to enhance graduate employability and patient safety, a form of active collaborative learning, Team-Based Learning (TBL) (Michaelsen, Parmelee, McMahon, & Levine, 2008) was implemented into the curriculum in 2009. Background Team-Based Learning is a highly designed teaching and learning strategy that maximises student preparation and participation, holding students accountable for their own learning before and during class. Students spend time in class solving authentic problems in essentially self-managed, high-performing, permanent teams (Michaelsen & Sweet, 2011). The TBL strategy consists of a repeating sequence of four phases (summarised in Table 1). In Phase 1, students undertake pre-learning using instructor provided resources prior to coming to class. These resources include clear intended learning outcomes for the topic, the first step in constructing teaching and learning activities (Biggs & Tang, 2011). In the readiness assurance process (Phase 2), students undertake and submit for grading a multiple-choice test of the topic’s core concepts. Then students take the same test in their 5 to 7 member team. In Phase 3, each team works together to solve the same significant, complex, authentic problems, applying knowledge acquired and demonstrated through Phases 1 and 2. The teacher then facilitates a discussion, with teams defending their position in an inter-team debate. In this process, students refine and extend their thinking, leading to deeper learning, improved problem solving skills, and positive personal and team behaviours (Haidet et al., 2012). Within teams, students also give and receive periodic formative and summative peer evaluations of each member’s contribution and team behaviours (Phase 4), encouraging reflection, self-management and improved team behaviours (Michaelsen & Sweet, 2011). Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 95 Table 1: Phases of Team-Based Learning PHASE 1 Pre-class learning PHASE 2 Readiness Assurance Process PHASE 3 Application of course concepts PHASE 4 Peer evaluation Independent student learning  Study materials provided, including instructor- set learning objectives Individual and team multi- choice test  Individual test answers submitted for grading  Teams take test and reach consensus by discussion  Team answers submitted and graded  Instructor feedback In class team assignment  Cooperative problem solving  Instructor- facilitated class discussion between teams Formative and summative Adapted from Currey, Eustace, et al. (2015) and Michaelsen and Sweet (2011). Team-Based Learning facilitates deep learning and optimises student engagement in a number of disciplines including nursing, medical, pharmacy, business, and education (Currey, Oldland, Considine, Glanville, & Story, 2015; Dunaway, 2005; Oldland, Currey, Considine, & Allen, 2017; Sisk, 2011). Team-Based Learning engages learners, and produces improved student learning outcomes (Dearnley, Rhodes, Roberts, Williams, & Prenton, 2018; Reimschisel, Minor, Herring, & Huang, 2017), including enhanced communication and team behaviours, and improved problem solving and critical thinking skills (Currey, Eustace, et al., 2015; Nieder, Parmelee, Stolfi, & Hudes, 2005; Park, Kim, Park, & Park, 2015). Medical and nursing educators perceive TBL to be superior to lecture-based education in student learning (depth, breadth, retention, and application of knowledge), student behaviours (self-directed and active learning, and peer-to-peer learning and teaching) and student skills (accountability, leadership, teamwork, communication, problem solving, collaboration, and teaching) (Oldland, Currey, Glanville, & Considine, 2012; Wu, Farquhar, & Compton, 2018). Postgraduate critical care students’ similarly report that TBL shifted them to more active learning strategies, deeper learning, improved critical thinking and problem solving, and better communication, including enhanced listening and feedback skills (Oldland et al., 2017). While students commonly perceive their TBL experience positively, and broadly identify development of numerous graduate learning outcomes, the role TBL may play in shaping their development of specific University Graduate Learning Outcomes is currently underexplored. Ascertaining the perceptions of postgraduate nursing students may provide further evidence of the role of TBL in the development of mature practice-ready graduates and may suggest areas for further development of graduate attributes across the curricula. Methods Aim The aim of this study was to describe postgraduate specialty nursing student perceptions of how Team-Based Learning contributed to the development of professional skills defined by the University’s Graduate Learning Outcomes (GLOs). The GLOs were: discipline-specific knowledge and capabilities, communication, digital literacy, critical thinking, problem solving, self-management, teamwork, and global citizenship. The definitions of each GLO are presented in Table 2. Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 96 Table 2: Definitions of Graduate Learning Outcomes (Deakin University, 2018) Discipline-specific knowledge and capabilities  appropriate to the level of study related to a discipline or profession Communication  using oral, written and interpersonal communication to inform, motivate and effect change Digital literacy  using technologies to find, use and disseminate information Critical thinking  evaluating information using critical and analytical thinking and judgement Problem solving  creating solutions to authentic (real world and ill-defined) problems Self-management  working and learning independently, and taking responsibility for personal actions Teamwork  working and learning with others from different disciplines and backgrounds Global citizenship  engaging ethically and productively in the professional context and with diverse communities and cultures in a global context Design A descriptive exploratory design was used to address the research aim. Approval to conduct the study was granted by the Human Research Ethics Committees at the study site (HEAG-H 173_2016). Setting and participants The study took place in the calendar years of 2016 and 2017 at a university in Melbourne, Australia. Participants were postgraduate nursing students undertaking the first year of the Master of Nursing Practice in either Intensive Care, Cardiac Care, Emergency Care or Critical Care. Students participate in TBL classes fortnightly during trimester, attending a total of 13 TBL classes for the year. These 177 students were invited to participate in the study on the final day of classes for 2016 and 2017. Written informed consent was granted by participants. Data collection Participation in the research involved the completion of: 1) a demographic survey combined with a GLO ranking exercise (10 minutes) and 2) a 300-word written reflection (20 minutes). Demographic survey and Graduate Learning Outcome ranking exercise Students who consented to participate were required to provide demographic information and rank each GLO according to the degree with which they perceived TBL aided the development of GLOs. Descriptions of each of these outcomes were provided for the students’ reference during completion of this ranking exercise. Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 97 Written reflection A written reflection is a pre-existing, mandatory assessment, worth 2.5 per cent, for all students in the course, irrespective of their involvement in this study. The primary purpose of the reflection assessment in the curricula is to assist in developing students’ reflective ability about how they learn and what they’ve learnt. Students are required to reflect on how TBL may have contributed to their personal learning style, team behaviours, and clinical and professional behaviours. Students are awarded the allocated marks if their work contains evidence of reflection, irrespective of whether their perceptions about TBL are positive, negative or neutral. Students are informed explicitly that their perceptions about TBL have no bearing on their mark or relationships with TBL teachers, and encouraged to reflect honestly. For the purposes of this research, students were additionally and specifically asked to reflect on whether or not they perceived their participation in TBL classes contributed to their acquisition of the University’s GLOs. Reflections of students who consented to participate were de-identified and provided for analysis; reflections of those who did not consent to participate were securely discarded. Reflections were analysed by a research fellow (SS) and teacher (GB) not involved in grading reflections. The Unit Chair (EO) was blinded to students who did and did not consent to participate in this study, and analysed reflections for marks as per usual processes. Data analysis Descriptive statistics were used to summarise the demographic data and the GLO ranking data using SPSS version 23 (IBM Corp, 2015). De-identified reflections were analysed using content analysis techniques (Elo & Kyngäs, 2008). Reflections were coded for the presence of positive or negative comments in relation to any of the eight GLOs. A positive reflection was defined as an instance where the participant expressed that their TBL experience provided opportunities to develop and/or positively influence the acquisition of the skills detailed in the definition of each GLO (Table 2). A negative reflection was defined as an instance where the participant expressed their TBL experience hindered or did not provide opportunities to develop, and/or impeded the acquisition of the skills defining a GLO. The University’s definitions of the GLOs (Table 2), were referred to consistently throughout this process to ensure accurate interpretation of the data. Notably, a single statement could incorporate mentions of multiple GLOs, given that aspects of each GLO potentially overlap. In instances where participants repeated similar sentiments in relation to a GLO the repeated instance was counted as another independent reflection. This decision was because repeated references to the development of a GLO were considered to signify particular importance. Inter-rater reliability was performed on four reflections by the primary investigator (JCu) and the research fellow (SS) with a 94 per cent agreement reached. Post discussion, the level of agreement was 100 per cent (Suen & Ary, 1989). The same inter-rater reliability process was completed between the research fellow and teacher (GB) with 96 per cent agreement reached, and after discussion 100 per cent agreement reached. Results Participant characteristics In total, 172 of the 177 eligible students consented to participate in this research and completed the demographic survey (Table 3), giving a response rate of 97.2 per cent. Of these, 156 (90.7%) students completed the entire GLO ranking exercise, and 144 (83.7%) students consented to submission of their written reflection for analysis. Most participants were female (n = 152, 88.4%), aged 25 – 34 years (n = 115, 66.9%), with a median of 4 years (Q1, Q3: 3,6) nursing experience. Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 98 Table 3: Participant Characteristics (N = 172) n % Female gender 152 88.4 Age (years) (n = 171) ≤ 24 30 17.4 25 – 34 115 66.9 35 – 44 19 11.0 45 – 54 6 3.5 55 – 65 1 0.6 Critical care area (n = 171) Intensive Care Unit 85 49.7 Coronary Care Unit 49 28.7 Cardiac Catheterisation Laboratory 1 0.6 Emergency Department 18 10.5 Combined Unit 18 10.5 Workplace (n = 170) Metropolitan public 76 44.7 Metropolitan private 75 44.1 Regional public 15 8.8 Regional private 2 1.2 Rural public 2 1.2 Rural private 0 0.0 Mdn Q1, Q3 Hours worked per week (n = 170) 32 30, 32 Note. Mdn = median; Q1, Q3: 25th, 75th percentiles. Rankings of Graduate Learning Outcome Student (n=156) rankings indicated TBL contributed to their acquisition of critical thinking (n = 90, 57.7%) the most, followed by problem solving (n = 56, 35.9%), and teamwork (n = 45, 28.8%). Conversely, students ranked TBL contributed the least to global citizenship (n = 103, 66.0%). Table 4 shows student perceptions of the ranked order in which they perceived TBL classes contributed to acquisition of the eight GLOs. Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 99 Table 4: Rank Order of Perceived Influence of Team-Based Learning on Acquisition of Graduate Learning Outcomes (n = 156) *Note: 1 = TBL supported this Graduate Learning Outcome the most, 8 = TBL supported this Graduate Learning Outcome the least. Critical thinking Problem solving Teamwork Communication Discipline- specific knowledge and capabilities Self- management Digital literacy Global citizenship Ranking* n (%) n (%) n (%) n (%) n (%) n (%) n (%) n (%) 1 90 (57.7) 15 (9.6) 23 (14.7) 8 (5.1) 13 (8.3) 3 (1.9) 2 (1.3) 1 (0.6) 2 35 (22.4) 56 (35.9) 23 (14.7) 28 (17.9) 14 (9.0) 3 (1.9) 0 (0.0) 0 (0.0) 3 12 (7.7) 35 (22.4) 45 (28.8) 30 (19.2) 22 (14.1) 10 (6.4) 1 (0.6) 2 (1.3) 4 12 (7.7) 24 (15.4) 32 (20.5) 46 (29.5) 23 (14.7) 18 (11.5) 2 (1.3) 1 (0.6) 5 3 (1.9) 18 (11.5) 17 (10.9) 32 (20.5) 55 (35.3) 20 (12.8) 7 (4.5) 2 (1.3) 6 2 (1.3) 7 (4.5) 9 (5.8) 11 (7.1) 21 (13.5) 78 (50.0) 14 (9.0) 12 (7.7) 7 1 (0.6) 1 (0.6) 7 (4.5) 0 (0.0) 8 (5.1) 22 (14.1) 81 (51.9) 35 (22.4) 8 1 (0.6) 0 (0.0) 0 (0.0) 1 (0.6) 0 (0.0) 2 (1.3) 49 (31.4) 103 (66.0) Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 100 Written reflections There were 2719 comments in written reflections by 144 students regarding TBL and the acquisition of GLOs. All 144 students made at least one positive reflection resulting in 2657 (97.7%) comments (Table 5). Of all positive reflections, self-management (n = 520, 19.6%) was the most common, followed by communication (n = 434, 16.3%), and teamwork (n = 431, 16.2%). Digital literacy received the lowest number of positive reflections (n= 19, 0.7%). Twenty-two students (15.3%) wrote 62 negative reflections about the influence of TBL on GLO acquisition. These 62 negative reflections comprised 2.3% of all comments analysed. The negative reflections were most commonly related to discipline-specific knowledge and capabilities (n = 21, 33.9%), communication (n = 12, 19.4%), and global citizenship (n= 10, 16.1%) (Table 5). All students who made negative comments had also made positive comments about TBL and GLO acquisition. In addition to the quantifying student reflections, a content analysis was performed. Findings in the form of a narrative summary of student reflections related to each GLO is presented in the following section. Table 5: Team-Based Learning Reflections for Graduate Learning Outcome Acquisition (N = 2657) Positive reflections (N = 2657) Negative reflections (N = 62) Graduate Learning Outcome n % Mdn Q1,Q3 n % Mdn Q1,Q3 Self-management 520 19.6 3 2, 5 8 12.9 0 0, 0 Communication 434 16.3 3 2, 4 12 19.4 0 0, 1 Teamwork 431 16.2 2 2, 4 8 12.9 0 0, 0 Critical thinking 358 13.5 2 1, 3 1 1.6 0 0, 0 Discipline-specific knowledge and capabilities 313 11.8 2 1, 3 21 33.9 1 0, 1 Problem solving 294 11.1 2 1, 3 0 0.0 0 0, 0 Global citizenship 288 10.8 1 1, 3 10 16.1 0 0, 1 Digital literacy 19 0.7 0 0, 0 2 3.2 0 0, 0 Note: Mdn = median; Q1, Q3: 25th, 75th percentiles. Self-management Students reported TBL improved their self-discipline and overall motivation to prepare for in- class activities. Students reflected that learning new material can be difficult, and TBL required some students to reconsider their approach to study. Students frequently reflected that participation in TBL meant that they needed to take responsibility for their own learning, by changing or reprioritising study behaviours. Moreover, several commented that TBL sessions had been so challenging and thought provoking, this motivated them to refine their own learning style. Failure motivated some students to continue with their personal study and find ways to reach a satisfactory level of understanding. Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 101 TBL has enabled me to reassess my learning method, and thus adopt new techniques in order to maximise my learning… I used to focus on areas that only interest me, but now I have come to accept learning is not all about comfort. TBL has challenged me to ensure I was accountable and organised by being prepared for each session, so I could get the most out of the sessions. Commonly, students reported motivation to complete pre-learning more thoroughly because of a sense of responsibility to their peers; they did not want to let their team down. With improved preparation for class, students felt motivated and confident to contribute to in-class discussions. They recognised that active participation could influence their peers and ensure all members of the team had the opportunity to learn the material. Typically I haven’t found furthering my own knowledge a compelling motivator to study, but the knowledge that the team was relying on me to have prepared was enough. Students reflected that TBL prompted them to become more self-aware. Team-Based Learning helped some students identify their strengths and weaknesses as learners and team members. Students’ own reflections about TBL performance, together with peer feedback, encouraged some students to be more honest with themselves about what they did not know. This sense of personal responsibility extended beyond the classroom and into the workplace. For some, TBL prompted reflection on personal workplace behaviours, and a personal motivation to improve how students carried themselves in the workplace. In the work place I wouldn’t receive such feedback, therefore I would have not realised that my personality traits could have contributed to suboptimal patient care and/or advocacy, so in that regard TBLs brought to surface my weakness in order for me to make a change. You become more aware of how you come across to others Improved self confidence in class and in clinical practice was reported by many students as a result of being involved in TBL. Students felt empowered by the knowledge they had acquired, more at ease communicating with each other, and in front of the larger student cohort. This typically translated to feeling more capable in their workplaces. My clinical practice has benefitted…I am less afraid of making alternative suggestions re care plans. I feel better about putting my ideas out in the open and less fearful of making a “silly suggestion. Clinically I feel so much more capable of applying the knowledge I have learnt to real patient scenarios. Not all students noted an improvement in confidence and motivation to learn. Some students reported a desire to return to traditional lecture delivery of new material, and felt that TBL did not suit their preferred learning style. A small number of students expressed the opinion that there was no benefit in group learning and those students preferred to remain independent. Communication Improved communication skills in the classroom and in the workplace were frequently mentioned in student reflections. For many students, TBL provided the opportunity to practice communicating confidently, clearly, and succinctly within a supportive team and classroom. Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 102 Students reflected on the positive impact on confidence in oral communication skills as a result of in-class TBL activities. TBL enhanced my public speaking skills as I had always been afraid of speaking publicly. It provided a nurturing environment in which I was not afraid to speak out loud, and a safe place where I could ask the questions I wanted to ask. For many students, the new confidence in classroom communication translated to the student’s clinical setting. Students reported feeling more capable of clearly articulating their opinions and recommendations for their patients to the multidisciplinary team. As a result of better communication skills, students reported becoming more effective advocates for their patients, more involvement in team discussions, and ultimately greater job satisfaction. Now when I need to speak to doctors or have an issue with my patient I am physically more confident, standing taller and making eye contact. I now feel more comfortable speaking louder and voicing my concerns clearly. I know that before this I often mumbled or felt that my communication was jumbled or unclear… Communication skills were improved for many students through the need to teach complex concepts to team members. Students were motivated to help their team members understand concepts, and so needed to consider how best to deliver the message. To improve communication, students closely observed others around them, critiqued positive attributes evident in their peers, and were motivated to test new strategies in class and within the workplace. In addition, the opportunity to speak aloud key concepts deepened students’ own understanding. Some considered the time constraints of TBL an advantage to prompt communication that was of greater precision and avoided conflict. …since completing TBL I have seen the enormous benefit that talking and questioning with peers has had on my own understanding. For some, TBL helped develop active listening skills and patience. Feedback from peers and a greater insight into personal capacities highlighted the need to stop and listen, and to encourage quieter members of the team to contribute to discussions. Some students recognised that they had a responsibility to ensure that all team members understood the content and had the opportunity to have questions answered. Working in a team, you have to be respectful of each other…having time to listen and not argue with reason, having patience, valuing time as we only had a few minutes. I always knew of the importance of active listening, but never realised that in my excitement to ‘get things right’ that I was not helping others to voice their opinions and learn. A very small number of students described poor functioning teams characterised by conflicts and hierarchies. For some students, this meant they felt silenced. Of those who explored this notion, many stated it was not resolved. Conflict sometimes arose between students from different clinical backgrounds (e.g. intensive care and emergency nursing). I likened TBL to being in a high school tutorial with a cohort of disruptive teenagers...I eventually felt that it was not worth saying anything in TBL, because of the 'established hierarchy' who would make decisions for the 'group', regardless of what you had to say. Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 103 Teamwork Students reported recognising the features of a high functioning team, the individual’s responsibility to the team, and developing a greater appreciation of how team diversity led to better decision making. Many appreciated the diverse backgrounds of their team members and quickly learned that ‘many heads were better than one’. Some students found group learning difficult and challenging because of having to work with people they otherwise might avoid. However, over time students reported becoming more patient and accommodating, adapting to team dynamics and learning to get the best out of everyone. TBL has maximised the way I work with the team; it has helped me understand that the best solutions or answers are the ones that the team has worked thoroughly on. It has helped me value the contribution made by every member of the team and not just my own. I have loved the team work aspect of TBL to be engaged in passionate, heated discussions to achieve an answer was so satisfying and a bit thrilling. Students recognised the link between TBL and the power of a team approach in the workplace to solve problems and effect patient quality and safety. Students recognised their individual roles and responsibilities, and felt accountable to their fellow team members. Some participants noted that TBL sessions helped moderate their behaviours during conflicts, disagreements and failure. For others, TBL provided an opportunity to develop and practice skills in leadership. I do believe TBL facilitated an easier transition into a new work place, enabling me to form working relationships with team leaders and the collaborative team faster. I began to sit back and try to encompass a role of facilitator, probing the quieter members to share their ideas, as it restrained me to not be as overbearing… I found myself display leadership skills by directing the team to meet time limits and keep focused in making consensus decisions. Conversely, some students relayed frustration that their teams did not encourage equal participation. At times this meant that quieter students felt ‘left behind’ and undervalued. I did not enjoy the experience of being undermined by students who were outspoken and opinionated… Critical thinking Students reported an improved capacity for critical thinking, judgement of data, and analysing information in their day-to-day practice within the workplace. Students, within their practice of specialist nursing, felt competent to question rationales, evaluate practice standards, and incorporate the principles of evidence-based practice. Greater abilities for critical thinking led many students to evaluate their own clinical practice and the performance of their colleagues. TBL made me crave for more information rather than settling with the information provided… …I can better approach situations in my clinical area in a more exploratory way, rather than having a limited perspective… Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 104 I am consistently relating (disease) conditions, decisions and potential problems back to pathophysiology and putting the whole picture together, which is very satisfying. The collaborative problem solving of authentic case scenarios in TBL enhanced students’ critical thinking skills and enabled deeper analysis. This was often described as being ‘pushed to think outside the box’ and understand ‘the bigger picture’. Students reported developing critical thinking when being asked to rationalise their answer. Some reflected that they enjoyed the challenge of disagreeing with their team members and felt this led to deeper learning. Many also reported experiencing unexpected ‘light bulb’ moments after discussions held in class. TBL was actually helpful in moulding my mind to think deeper and not to only rely on the surface of the concepts, it’s like helping us connect the dots but in a different way which I think I need because I felt like it forces me to think outside of the box or outside of the pre-learning material for that matter. The TBL process mimics the clinical environment thought processes… Discipline-specific knowledge and capabilities The enhanced understanding of advanced concepts relevant to specialist nursing practice as a result of TBL was reported by many students. Students made links between theoretical knowledge, confidence as a clinician, and an improved ability to advocate for patients and effective patient care. I now have the knowledge to stand behind my answer and perhaps guide the course of treatment for my patient. I feel better equipped to ask questions and seek clarification. TBL has empowered me to have a say about the treatment plan for my patient. The knowledge and other professional skills developed through participation in TBL were reported to have improved students’ ability to deliver an effective handover to oncoming staff; a key requirement in healthcare to ensure safe patient care. Some students noted that improved specialist knowledge increased their capacity to educate and develop rapport with patients and relatives. Notably, students observed becoming a valuable resource for colleagues, even those who held senior positions. The realistic case scenarios used in TBL were recognised as rich opportunities for learning, prior to experience in the clinical setting. Exciting ‘light bulb moments’ were described where students suddenly grasped a challenging concept during a TBL class. Students attributed much of their growth and depth of knowledge to peer learning during TBL. I feel there is value in making judgement errors during hypothetical scenarios so that I can learn from my mistakes before moving into clinical practice. Other students expressed frustration with the teacher-facilitated discussion, particularly understanding the rationales for the correct and incorrect responses. Several described this as ‘unnecessarily drawn out’ and repetitive, which resulted in inattention and a loss of motivation, possibly impeding learning. I sometimes found listening to other peers challenging…it would sometimes further confuse my understanding of a topic. Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 105 I felt like the painstaking process of the class chats were unnecessarily drawn out and not enjoyable. Problem solving Challenging, authentic case scenarios required students to become more innovative and open to new perspectives. Students reported having to develop strategies to solve problems presented in TBL. They learnt to work together, and use the team diversity, to define the problem according to underlying principles and concepts. Some reflected they had learned patience. Others learned to take risks, even if they were not completely confident with their answer or understanding of a concept, many commented they ‘had a go’ at explaining it to their team or the whole class. TBL has challenged me and opened my mind into a new way of problem solving…getting down to the ‘but why’ not just settling for the ‘this is how we do it. Importantly, many reflected that they used new problem-solving skills immediately in their clinical practice. I learned to put aside some time before the ward rounds at work and think about actual and potential patient problems, strategies/measures that could overcome these problems and then discuss with the medical team during the round… Digital literacy Only 6 students (9.4%) reflected that TBL positively contributed to their development of skills in digital literacy. Students typically expressed that they gained greater appreciation for technology and its role in learning. Some noted they had become more efficient and resourceful when searching for electronic resources to complement the TBL topic. Other students stated that TBL did not assist with their digital literacy but did not offer reasons for this. Global citizenship Team-Based Learning participation had made some students more open, adaptive and accepting of others. For some, this translated to clinical settings with students admitted to becoming more trusting and respectful of their colleagues and their colleagues’ experiences. Students who had previously been apprehensive of clinical advice given by their colleagues now conceded there were often ‘many ways to skin a cat’. Students described becoming more open to collaborate with others beyond their immediate clinical area. The value of diversity in team performance became evident to many students through participation in TBL. Students recognised that learning and academic success relied on engaging with all team members. There was an acknowledgement that differences were advantageous and ought to be nurtured and explored. Many students reflected that their team was highly inclusive; a sense of comradery was often evident and they patiently supported each other while trying to grasp difficult concepts. The group dynamic also brought us together as a diverse and colourful bunch of individuals. Everyone had something important to offer, and everyone had a different style of engaging and participating. TBL smacked me in the face with a prejudice…against other people’s opinions I didn’t even know that I had. So mostly I appreciated TBL in that it made me face the fact that just because somebody arrives at an answer in a different way Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 106 doesn’t mean that his or her way is wrong. It just means that it is different. And I really grew to appreciate these different ways of knowing and understanding that each person brought to the team. A few students explained that a minority of groups were exclusive. More outspoken members dominated the discussions and decisions, negatively affecting the overall experience of some. Discussion The aim of this study was to evaluate postgraduate specialty nursing students’ perceptions of how TBL influenced the acquisition of professional skills defined by the university’s eight GLOs. The major findings of this study were that critical thinking and problem-solving were GLOs ranked highest by students as being influenced by TBL; digital literacy and global citizenship were the GLOs ranked lowest as being influenced by TBL; and self-management, communication, and teamwork were GLOs with the highest number of positive student reflections. The study participants were postgraduate critical care nursing students from one university; however, with characteristics typical of the critical care nursing population. Current national data shows that 89.1 per cent of Australian nurses and midwives are women (NMBA, 2017), comparable to the 88.4 per cent female sample reported in this study. Nationally, 24.6 per cent of nurses and midwives are aged 25-34 years (NMBA, 2017), compared with two-thirds (66.9%) of participants in this study. This age difference, along with the years of nursing experience of the participants, reflects a common career trajectory in critical care nursing, whereby specialisation is undertaken at a relatively early career stage. Team-Based Learning requires students to apply learned knowledge to solve significant, authentic and complex scenarios individually and within a team. Participants in our study reported that TBL most strongly influenced development of critical thinking and problem solving skills which are desired outcomes of this teaching strategy. Improvement in test scores in students undertaking TBL compared to traditional classrooms may be considered a proxy measure of problem solving and critical thinking. A recent systematic review of TBL in health professions education examined 85 studies related to learning outcomes and showed that student test scores either remained static (6 studies) or improved (79 studies) (Reimschisel, Minor, Herring, & Huang, 2017). In nursing, a randomised controlled trial by Kim, Song, Lindquist and Kang (2016) showed that third year nursing students assigned to TBL demonstrated advanced performance in thinking and problem-solving skills compared to control students who were assigned to traditional didactic teaching methods. The association between TBL and higher test scores has also been shown in several other nursing studies using quasi-experimental or pre-post test methods (Bouterie Harmon & Hills, 2015; Cheng, Liou, Tsai, & Chang, 2014; Mennenga, 2013). Critical thinking and problem solving skills are underpinned by knowledge (Pitt, Pwois, Levett-Jones, & Hunter, 2015); therefore the higher levels of knowledge acquired using TBL as a teaching strategy is a key factor in developing critical thinking and problem solving skills in graduates. Student and faculty perceptions that TBL develops critical thinking and problem solving skills have been reported in the literature internationally in numerous cohorts across various disciplines (Currey, Eustace et al., 2015; Currey, Oldland et al., 2015; Deardorff, Moore, Borges, & Parmelee, 2010; McCubbins, Paulsen, & Anderson, 2018; Nieder et al., 2005; Oldland, Currey, Considine, & Allen, 2017). Participants in the current study ranked these two skills as those most strongly developed by TBL, a finding consistent with the literature about student perceptions of TBL. Participants, as practicing clinicians, are very aware of their responsibility to critically think and problem solve to make multiple complex decisions in response to conditions of uncertainty and time pressure (Arslanian-Engoren & Scott, 2014). Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 107 This role awareness may have contributed to participants ranking these two GLOs highest. This is a pleasing finding as these learning outcomes are identified as the number one and two skills most required in the workforce of 2020 by The World Economic Forum (2016). The GLO of self-management represented the highest proportion of positive reflections in this study. Self-management is defined as ‘working and learning independently, and taking responsibility for personal actions’ (Deakin University, 2018). A hallmark feature of TBL is that students take personal responsibility for their learning, specifically pre-class preparation, and are accountable to their team (Michaelsen et al., 2008). Team-Based Learning has been shown to increase student engagement demonstrated through significant increases in time spent online viewing learning materials (Whittacker, 2015), and time spent studying outside of scheduled classes (Bouterie Harmon & Hills, 2015). The processes of TBL are designed to motivate students to take more responsibility for their learning, with students reporting an improved attitude to learning when taught using TBL (Park, Kim, Park, & Park, 2015). Implementation of TBL has been associated with significant increases in self-directed learning scores, thereby promoting lifelong professional learning strategies (Cheng, Liou, Hsu, Pan, Liu, & Chang, 2014). Students recognising TBL as supporting the development of self-management is a positive finding given the autonomous nature of critical care nurses’ decision-making and patient safety outcomes arising from those decisions. Interestingly, 50 per cent of participants gave self-management a low ranking despite it featuring so frequently in reflections. This GLO, and the way it is defined, may not have resonated with students as a primary purpose or outcome of TBL, given that most of their TBL interactions occur in a team context. It is evident that while students did not identify self-management as a high- ranking GLO, the words they chose to reflect on how TBL may have contributed to their personal learning style, team behaviours, and clinical and professional behaviours, demonstrated characteristics of the GLO. Students reflected on their TBL experience in promoting strong teamwork and high-level communication skills. They described a sense of personal responsibility to teammates to ensure all members of the team had an adequate level of understanding, requiring self- management to communicate information effectively. Team-Based Learning has been linked with improved interpersonal and team-based behaviours and significant increases in team efficacy, team-adaptability skills, and team-interpersonal skills (Huitt, Killins, & Brooks, 2014; Park et al., 2015). However, a recent systematic review showed that of 13 studies that examined whether or not learners’ attitudes about team work changed after experiencing TBL, seven showed significant improvements and six studies showed little or no change in learner attitudes about team work (Reimschisel et al., 2017). Participants in this study who commented on teamwork and communication GLOs recognised that TBL improved individual abilities to work effectively in diverse teams, and that effective communication and team- based behaviours lead to highly functioning teams. It is possible that there was an element of social desirability in selecting critical thinking and problem solving as the highest ranking GLOs acquired through participation in TBL. The complex case scenarios and in depth conversations within teams, and between teams and class facilitators very clearly and explicitly focus on developing these two attributes. Thus it may have influenced students’ perceptions of the importance of TBL in developing these two GLOs. It is likely that students, as critical care nurses, were seeking to develop critical thinking and problem solving in order to provide safe patient care as these attributes are highly valued in healthcare organisations and by the nursing profession. Further, it is also possible that students remembered the processes of critically thinking and problem solving as their dominant memories of TBL. Digital literacy and global citizenship were the GLOs least influenced by TBL, as ranked by students. Team-Based Learning does not aim to develop digital literacy as defined in this study (Table 2), so this is not a surprising finding. It is possible that students perceived they Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 108 already possessed a high level of digital literacy through everyday private use of technology, together with the expectations of digital literacy in the academic arena and in clinical practice. Global citizenship is defined as ‘engaging ethically and productively in the professional context and with diverse communities and cultures in a global context’ (Deakin University, 2018). It is possible that not all students fully understood the GLO or recognised the role TBL may have had in developing this GLO. Students who did comment on this GLO clearly recognised the development of global citizenship in the ability to work effectively in diverse teams that include members from different employing hospitals, undertaking different specialty programs (emergency nursing, intensive care nursing and cardiac nursing) and from different cultural and social backgrounds, reflecting the diverse nature of the nursing workforce. Working with and within diverse communities and cultures is a usual part of clinical practice, and therefore students may not have seen these elements of the global citizenship GLO as an area of development. There are no studies identified in the literature that specifically explore perceived acquisition of GLOs (which would vary between institutions), so it is not possible to compare our findings with other study results specific to this GLO. It is arguable that aspects of ‘teamwork’ and ‘communication’ are common to ‘global citizenship’ and may have been acknowledged in the rankings ascribed to these other learning outcomes. Most students reported positive experiences with TBL in their written reflections; however, the small number of negative reflections about TBL also require exploration. Specifically, some students reported that they disliked TBL due to an attachment to passive lecture-style teaching and a dislike of group learning. These findings were not surprising; TBL is often resisted in the implementation phase as it is markedly different to the traditional learning styles that students may be used to (Frame et al., 2015; Michaelsen & Sweet, 2012). For most of our students, TBL was a new construct reliant on a significant shift in thinking around learning processes. In other studies (Frame et al., 2015; Strayer, 2012) some students preferred passive learning styles over group-oriented active learning styles. Frame et al. (2015) investigated pharmacy students’ perceptions of TBL versus traditional lecture-based learning and found that students were more likely to prefer passive learning styles if they were exposed to this prior to participating in TBL. Therefore, timing may impact student perceptions of TBL. Our students are postgraduate level students and it is possible that they were accustomed to passive learning styles prior to commencing this course. It is also possible that preconceived notions of what ‘group work’ entails, fuelled apprehension around TBL processes for some students (Fink, 2002). We respect that students will always have individual needs and preferences, and acknowledge that it may take time for this learning style to be embraced. Clear student expectations are set in the beginning of the course about why and how TBL as a form of active learning will be used. Of note, TBL is one of several forms of learning in our course; no doubt other students dislike other forms too. Our students participated in TBL for 13 classes over 7 months, and it may take longer than this for some students to engage in TBL and appreciate its benefits. There were also reports of poor team functioning secondary to conflict and hierarchical issues and student inattention. Though this related to a very small number of students, the suggestion that some students felt dominated, undervalued, and discouraged from participating in TBL is very concerning as this may have impeded learning and damaged their self-esteem. It is worth noting that embedded in TBL is peer evaluation twice per trimester of study where such feedback can be given and addressed. While unfavourable, conflict within teams is sometimes unavoidable, and similar challenges may be faced in the workforce (Frame et al., 2015). It is disappointing that some students reported that conflicts were unresolved. According to Tuckman (1965), all teams go through the stages of forming, storming, norming and performing; perhaps some individuals perceived their team had not progressed past storming. Reports of unresolved conflict suggest that our future cohorts may benefit from further education and support around conflict resolution and team building exercises prior to the formal commencement of TBL sessions (Frame et al., 2015). In doing Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 109 so, these exercises may also optimise students’ problem solving, teamwork, and communication skills. Other research has also demonstrated that the student experience of working within a team may improve over time (Parmelee et al., 2009). As per recommendations, academics have an active role in team composition to ensure diversity in experience, preferences, learning styles and personality types (Frame et al., 2015). Nevertheless, not all these characteristics are evident to the teacher or student at the time of team formation. The comment around student inattention brings to bear responsibility by three parties to address: 1) the facilitator to ensure all students in the room remain engaged; 2) that the team manages its own members to engage in discussion; 3) that the students themselves lead the discussion to areas of their own learning need and/or recognise that other students have different learning needs. The latter is a key part of TBL – recognising and respecting the learning needs of peers. Future work readiness and the necessity of graduates developing transferable skills within the discipline specific knowledge and skills of their university degree (Oliver, 2011, 2015) has been a topic of commentary for the last decade. While the context for this study was postgraduate critical care nurses preparing for practice in healthcare environments, the GLOs we sought to develop are broad, generic and essential for almost any job in a competitive and changing market. Recent reports have highlighted the skills most needed in the workforce to meet the requirements of jobs by 2020 and beyond (World Economic Forum, 2016). A report published by The Foundation for Young Australians (FYA) (2017a) contends that young workers need ‘enterprise’ skills to navigate complex careers across a range of industries and professions in a rapidly changing job environment. These skills include problem solving, digital literacy, teamwork and communication. In another FYA report (2017b), it is predicted that by the year 2030 we will on average, spend double the time at work solving problems, spend 41 per cent more time on critical thinking and judgment, and utilise verbal communication and interpersonal skills for 7 hours a week each (up 17%). Educational programs have an obligation to design teaching and learning activities that contribute to the development of these skills. These ‘enterprise’ skills are intentionally developed through TBL. There are some limitations that should be considered when interpreting the study findings. First, this study focused on student perceptions of GLO acquisition so are self-reported data. Second, there were two data collectors (SS and GB) and although rigorous inter-rater reliability was conducted, it is possible that there were differences in the judgements of the researchers given the subjective nature of the data. Third, when the university definitions of the GLOs are considered, there are some areas of overlap which may have influenced student responses. Further, although students were provided with the definitions of each GLO, there was no way to verify that students read and understood the GLO definitions. Finally, students were openly and actively encouraged to be as honest as possible and reminded that their comments would be de-identified and have no bearing on their grades. However, there is a risk of social desirability and reporting bias that may have influenced students to articulate what they thought the researchers want to hear. Conclusions A high proportion of postgraduate critical care nursing students perceive that TBL is beneficial to their development and acquisition of the university’s GLOs. Critical thinking, problem solving, and self-management were strongly attributed to participating in TBL classes. Digital literacy and global citizenship were the GLOs that students perceived were least acquired by TBL participation. Student reflections were largely positive suggesting they perceive that TBL has a positive effect on GLO acquisition. As an active learning strategy, TBL is instrumental in embedding so called soft skills which are known to be vital to the success of graduates in any workplace. The results of this work strongly suggest that TBL is advantageous to learning discipline specific knowledge, and in the development of key Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 110 professional attributes vital for practice in healthcare environments. In doing so, TBL is a key educational experience for our students to meet essential nursing professional practice standards and the expectations of the healthcare workforce. Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 111 References Arslanian-Engoren, C., & Scott, L. D. (2014). Clinical decision regret among critical care nurses: A qualitative analysis. Heart and Lung: Journal of Acute and Critical Care, 43(5), 416–419. doi: 10.1016/j.hrtlng.2014.02.006 Australian College of Critical Care Nurses. (2015). Practice standards for specialist critical care nurses. Biggs, J., & Tang, C. (2011). Teaching for quality learning at university: What the student does (4th ed.). Maidenhead, England: McGraw Hill: Society for Research into Higher Education & Open University Press. Bouterie Harmon, R., & Hills, R. L. (2015). Transforming psychiatric mental health nursing education with Team Based Learning. Archives Of Psychiatric Nursing, 29(6), 413. doi:10.1016/j.apnu.2015.06.014 Cheng, C., Liou, S., Hsu, T., Pan, M., Liu, H., & Chang, C. (2014). Preparing nusing students to be competent for future professional practice: Applying the Team-Based Learning teaching strategy. Journal of Professional Nursing, 30(4), 347–356. Cheng, C., Liou, S., Tsai, H., & Chang, C. (2014). The effects of team-based learning on learning behaviors in the maternal-child nursing course. Nurse Education Today, 34(1), 25–30. doi:10.1016/j.nedt.2013.03.013 College of Emergency Nursing Australasia. (2013). Practice standards for the emergency nursing specialist. Retrieved from http://www.cena.org.au/wp- content/uploads/2014/10/Practice_Standards_for_the_Emergency_Nurse_Specialist_J une_14.pdf Currey, J., Eustace, P., Oldland, E., Considine, J., Glanville, D., & Story, I. (2015). Developing professional attributes in critical care nurses using Team-Based Learning. Nurse Education in Practice, 15, 232–238. doi: 10.1016/j.nepr.2015.01.011 Currey, J., Oldland, E., Considine, J., Glanville, D., & Story, I. (2015). Evaluation of postgraduate critical care nursing students' attitudes to, and engagement with, Team- Based Learning: A descriptive study. Intensive & Critical Care Nursing, 31(1), 19–28. doi: 10.1016/j.iccn.2014.09.003 Deakin University. (2018). Student Manual. Faculty of Health. Version 2.0. Geelong: Deakin University. Retrieved 23 May 2018 from http://www.deakin.edu.au/__data/assets/pdf_file/0006/1350348/Health-Faculty- Student-Manual-2018-Final-V2.pdf Deardorff, A., Moore, J., Borges, N., & Parmelee, D. (2010). Assessing first year medical student attitudes of effectiveness of team-based learning. Medical Science Educator, 20(2), 67–72. Retrieved from http://www.iamse.org/mse-article/assessing-first-year- medical-student-attitudes-of-effectiveness-of-team-based-learning/ Dearnley, C., Rhodes, C., Roberts, P., Williams, P., & Prenton, S. (2018). Team based learning in nursing and midwifery higher education; a systematic review of the evidence for change. Nurse Education Today, 60, 75–83. doi: 10.1016/j.nedt.2017.09.012 Dunaway, G. A. (2005). Developments: Adaption of team learning to an introductory graduate pharmacology course. Teaching and Learning in Medicine, 17(1), 56–62. doi: 10.1207/s15328015tlm1701_10 Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115. doi: 10.1111/j.1365-2648.2007.04569.x Fink, D. L. (2002). Beyond small groups: Harnessing the extraordinary power of learning teams. In L .K Michaelsen, A.B Knight, & L. Dee Fink (Eds), Team-based learning : A transformative use of small groups (pp. 3–27). Westport Connecticut, London: Praeger. Frame, T. R., Cailor, S. M., Gryka, R. J., Chen, A. M., Kiersma, M. E., & Sheppard, L. (2015). Student perceptions of team-based learning vs traditional lecture-based learning. American Journal of Pharmaceutical Education, 79(4), 51. http://www.cena.org.au/wp-content/uploads/2014/10/Practice_Standards_for_the_Emergency_Nurse_Specialist_June_14.pdf http://www.cena.org.au/wp-content/uploads/2014/10/Practice_Standards_for_the_Emergency_Nurse_Specialist_June_14.pdf http://www.cena.org.au/wp-content/uploads/2014/10/Practice_Standards_for_the_Emergency_Nurse_Specialist_June_14.pdf http://www.deakin.edu.au/__data/assets/pdf_file/0006/1350348/Health-Faculty-Student-Manual-2018-Final-V2.pdf http://www.deakin.edu.au/__data/assets/pdf_file/0006/1350348/Health-Faculty-Student-Manual-2018-Final-V2.pdf Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 112 Haidet, P., Levine, R., Parmelee, D., Crow, S., Kennedy, F., Kelly, P., . . . Richards, B. (2012). Guidelines for reporting team-based learning activities in the medical and health sciences education literature. Academic Medicine, 87(3), 292–299. doi: 10.1097/ACM.0b013e318244759e Henneman, E. A., Gawlinski, A., Blank, F. S., Henneman, P. L., Jordan, D., & McKenzie, J. B. (2010). Strategies used by critical care nurses to identify, interrupt, and correct medical errors. American Journal of Critical Care, 19(6), 500–509. doi: 10.4037/ajcc2010167 Huitt T., Killins A., & Brooks, W. (2014). Team-based learning in the gross anatomy laboratory improves academic performance and students’ attitudes toward teamwork. Anatomical Sciences Education, 8(2), 95–103. doi:10.1002/ase.1460 IMB Corp. (2015). IBM SPSS statistics for windows, version 22.0. Armonk, NY: IMB Corp. Kim, H., Song, Y., Lindquist, R., & Kang, H. (2016). Effects of team-based learning on problem-solving, knowledge and clinical performance of Korean nursing students. Nurse Education Today, 115. doi:10.1016/j.nedt.2015.12.003 McCubbins, O., Paulsen, T., & Anderson, R. (2018). Examining student perceptions of their experience in a TBL formatted capstone course. Journal of Agricultural Education, 59(1), 135–152. doi:10.5032/jae.2018.01135 Mennenga, H.A (2013). Student engagement and examination performance in a team- based learning course. Journal of Nursing Education,52(8),475–9. Michaelsen, L., Parmelee, D., McMahon, K., & Levine, R. (2008). Team-based learning for health professions education: A guide to using small groups for improving learning. Sterling, Virginia: Stylus Publishing. Michaelsen, L. K., & Sweet, M. (2011). Team‐based learning. New Directions for Teaching and Learning, 2011(128), 41–51. doi: 10.1002/tl.467 Michaelsen, L. K., & Sweet, M. (2012). Fundamental principles and practices of team- based learning. Team-based learning for health professions education: A guide to using small groups for improving learning, 9-34. Nieder, G. L., Parmelee, D. X., Stolfi, A., & Hudes, P. D. (2005). Team-based learning in a medical gross anatomy and embryology course. Clinical Anatomy, 18(1), 56–63. doi: 10.1002/ca.20040 Nursing and Midwifery Board of Australia (NMBA). (2016). Registered Nurse standards for practice (4th ed.). Dickson, Australian Capital Territory: Nursing and Midwifery Board of Australia. Retrieved 18 January 2018 from http://www.nursingmidwiferyboard.gov.au/Codes-Guidelines-Statements/Professional- standards/registered-nurse-standards-for-practice.aspx Nursing and Midwifery Board of Australia. (2017). Nursing and Midwifery Board of Australia Registrant Data, 1 October 2017 – 31 December 2017 (4th ed.). Dickson, Australian Capital Territory: Nursing and Midwifery Board of Australia. Retrieved 23 May 2018 from http://www.nursingmidwiferyboard.gov.au/About/Statistics.aspx Oldland, E., Currey, J., Considine, J., & Allen, J. (2017). Nurses’ perceptions of the impact of Team-Based Learning participation on learning style, team behaviours and clinical performance: An exploration of written reflections. Nurse Education in Practice, 24, 62–69. doi: 10.1016/j.nepr.2017.03.008 Oldland, E., Currey, J., Glanville, D., & Considine, J. (2012). Industry educators’ perceptions of student outcomes arising from Team-Based Learning. Paper presented at the Higher Education Research and Development Conference, Hobart, Tasmania. Oliver, B. (2011). Assuring graduate outcomes. 2015 (November 22), Support for the original work was provided by the Australian Learning and Teaching Council Ltd, an initiative of the Australian Government. Retrieved from http://www.assuringlearning.com/resources/Assuring_graduate_outcomes_ALTC_Goo d_practice_report.pdf http://www.nursingmidwiferyboard.gov.au/Codes-Guidelines-Statements/Professional-standards/registered-nurse-standards-for-practice.aspx http://www.nursingmidwiferyboard.gov.au/Codes-Guidelines-Statements/Professional-standards/registered-nurse-standards-for-practice.aspx http://www.nursingmidwiferyboard.gov.au/About/Statistics.aspx http://www.assuringlearning.com/resources/Assuring_graduate_outcomes_ALTC_Good_practice_report.pdf http://www.assuringlearning.com/resources/Assuring_graduate_outcomes_ALTC_Good_practice_report.pdf Currey, J., Sprogis, S.K.., Burdeu, G., Considine, J., Allen, J., & Oldland, E. (2018). Students perceive Team-Based Learning facilitates development of graduate learning outcomes and professional skills. Journal of Teaching and Learning for Graduate Employability, 9(1), 93-113. 113 Oliver, B. (2015). Redefining graduate employability and work-integrated learning: Proposals for effective higher education in disrupted economies. Journal of Teaching and Learning for Graduate Employability, 6(1), 56–65. doi: 10.21153/jtlge2015vol6no1art573 Park, H., Kim, C., Park, J., & Park, E. (2015). Effects of team-based learning on perceived teamwork and academic performance in a health assessment subject. Collegian, 22, 299–305. doi: 10.1016/j.colegn.2014.05.001 Parmelee, D. X., DeStephen, D., & Borges, N. J. (2009). Medical students’ attitudes about team-based learning in a pre-clinical curriculum. Medical Education Online, 14(1), 4503. Pitt, V., Powis, D., Levett-Jones, T., & Hunter, S. (2015). The influence of critical thinking skills on performance and progression in a pre-registration nursing program. Nurse Education Today, 35(1), 125–131. Reimschisel, T., Minor, T. J., Herring, A. L., & Huang, J. (2017). A systematic review of the published literature on team-based learning in health professions education. Medical Teacher, 39(12), 1227–1237. doi: 10.1080/0142159X.2017.1340636 Strayer, J. F. (2012). How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environments Research, 15(2), 171–193. Sisk, R. J. (2011). Team-based learning: Systematic research review. Journal of Nursing Education, 50(12), 665-669. doi: 10.1111/j.1744-6198.2009.00162.x Suen, H. K., & Ary, D. (1989). Analyzing quantitative behavioral observation data. Hillsdale, New Jersey: Lawrence Erlbaum Associates. The Foundation for Young Australians. (2017a). The new basics: Big data reveals the skills young people need for the new work order. Sydney. Retrieved from https://www.fya.org.au/our-research/#report-downloads The Foundation for Young Australians. (2017b). The new work smarts:Thriving in the new work order. Sydney. Retrieved from https://www.fya.org.au/our-research/#report- downloads Tuckman, B. W. (1965). Developmental sequence in small groups. Psychological Bulletin, 63(6), 384–399. http://dx.doi.org/10.1037/h0022100 Whittaker, A. (2015). Effects of team-based learning on self-regulated online learning. International Journal Of Nursing Education Scholarship, 12(1), 1–10. doi:10.1515/ijnes-2014-0046 World Economic Forum. (2016). The future of jobs: Employment, skills and workforce strategy for the fourth industrial revolution Global Challenge Insight Report. Retreived from http://www3.weforum.org/docs/WEF_Future_of_Jobs.pdf World Health Organization (WHO). (2011). WHO patient safety curriculum guide: Multi- professional edition. Geneva, Switzerland: World Health Organization. Retrieved from http://apps.who.int/iris/bitstream/handle/10665/44641/9789241501958_eng.pdf;jsessio nid=CC3F1261EDD010BB38D416C4283AC4A1?sequence=1 Wu, S., Farquhar, J., & Compton, S. (2018). Why do Team-Based Learning educators use TBL? TAPS 2018, 3(1), 38-41. doi: https://doi.org/10.29060/TAPS.2018-3-1/SC1040 Yorke, M. (2006). Learning and employment Series 1: Employability in higher education: what it is - what it is not. The Higher Education Academy. Retrieved from http://www.voced.edu.au/content/ngv%3A16446 https://www.fya.org.au/our-research/#report-downloads https://www.fya.org.au/our-research/#report-downloads https://www.fya.org.au/our-research/#report-downloads http://www3.weforum.org/docs/WEF_Future_of_Jobs.pdf http://apps.who.int/iris/bitstream/handle/10665/44641/9789241501958_eng.pdf;jsessionid=CC3F1261EDD010BB38D416C4283AC4A1?sequence=1 http://apps.who.int/iris/bitstream/handle/10665/44641/9789241501958_eng.pdf;jsessionid=CC3F1261EDD010BB38D416C4283AC4A1?sequence=1 http://www.voced.edu.au/content/ngv%3A16446