Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 26 Using successful graduates to improve the quality of undergraduate nursing programs Geoff Scott 1 , Esther Chang 2 and Leonid Grebennikov 1 g.scott@uws.edu.au; e.chang@uws.edu.au and l.grebennikov@uws.edu.au 1 Office of Planning and Quality, University of Western Sydney 2 School of Nursing and Midwifery, University of Western Sydney Abstract This paper outlines the context and focus of the late 2008 survey of 45 early career nursing graduates working in public hospitals in an outer-urban area of Sydney who were identified by their supervisors as performing successfully. It gives an overview of the key quantitative and qualitative results for 2008 compared with the results of earlier studies of successfully performing nurses and other professionals in the first 3-5 years of their career. Based on this analysis it makes a range of recommendations on how to optimise the quality and relevance of the learning design, support and assessment systems experienced by undergraduate nursing students. The study confirms that, in addition to possessing a high level of technical competence, it is a particular combination of personal, interpersonal and cognitive capabilities which characterises the most effective performers. The most productive approaches to developing these capabilities focus on: consistent two-way links between theory and practice; the availability of responsive, committed academic staff with current clinical experience; provision of clear direction and integration between different units of study; the systematic use of case- based learning, simulations, clinical placements and associated assessment tasks which are “real-world” focused, integrated and problem-based, and which concentrate on the key capabilities identified as counting most for successful early career practice in this and parallel studies. Key words: competencies; capabilities; surveying; successful graduates; learning design Introduction Need The 2008 Review of Australian Higher Education (the so called Bradley Review) reports: Submissions from business and industry, in particular, discussed the types of skills sought in graduates. Technical skills and generic employability skills (such as communication and language skills) were considered to be of equal importance. (Department of Education, Employment and Workplace Relations [DEEWR], 2008, p.210). However, understanding exactly what is meant by concepts like “employability skills” as they apply to the professions taught at universities, determining exactly how to identify the capabilities and competencies most telling for successful early career practice and then effectively shaping and assessing them in context remains relatively unexplored. mailto:g.scott@uws.edu.au mailto:e.chang@uws.edu.au mailto:l.grebennikov@uws.edu.au http://pubapps.uws.edu.au/teldir/schlprocess.php?SNFCH Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 27 Building on the ideas of John Dewey (1933), the work of Gonczi, Hager and Oliver (1990) and a successful program developed for Australia’s Skill Olympians in the mid 1990s (Scott & Saunders, 1995) a series of pilot studies involving some 200 successful early career graduates in nine professions 1 was undertaken by the University of Technology, Sydney (UTS) between 2000 and 2005 (Rochester, Kilstoff & Scott, 2005; Scott & Yates, 2002; Vescio, 2005). Apart from these studies we have been able to locate little research that specifically uses successful graduates to identify the capabilities that count most for effective early career practice or to provide feedback on what aspects of their university experience have proven to be most (and least) productive in preparing them for their first 3-5 years of professional practice, especially at a broad regional, state or national level. Even fewer studies have explicitly used the findings to “backward map” (Elmore, 1979) to test and enhance the relevance and validity of curriculum and assessment design. One of the most valuable resources for adult learning is having access to the experience of a “fellow traveller” further down the same professional path (Tough, 1979; Scott, 2008). External advisory committees for course development are often comprised of professionals who, by virtue of their seniority and role, are unlikely to be in direct contact with the day to day practicalities and realities of junior staff. Studies of successful graduates using the proven professional capability and learning frameworks applied in this and the earlier studies provide a double benefit. It builds in relevance to new students because they know that what they are to learn and how they are to learn it has proven to be of direct significance to their successful graduate peers. Furthermore, it provides a useful mechanism to validate and situate the professional capability and competency frameworks developed by advisory committees and accreditation panels. The Bradley Review and Minister Julia Gillard (2009) have confirmed that participation and completion levels in Australian Higher Education are to increase, with a particular focus on low socio-economic status student access and success. For this objective to be achieved educators need to optimise the focus, relevance, efficiency and productivity of the learning which undergraduates experience in our universities. The Nursing profession is a key foundation for the future of Australia. Producing imaginative, effective and successful young graduates, people able to take up leadership roles over the next decade, is critical to the future of this country with its expanding health challenges, an aging population and the need to deal productively and promptly with the twin pressures of the financial and climate crises. The Business Council of Australia (BCA) has emphasised that a sound health care system populated by highly capable staff is a key economic as well as social issue and that higher education is an investment not a cost (Business Council of Australia, 2009). It is for all of the above reasons that a team of researchers came together to work with key senior figures in the Sydney West Area Health Service (SWAHS) to test the potential for scaling up the pilot studies of successful nurses undertaken by the UTS to a regional, state or national level. This paper reports on the results of phase one of the scale up strategy – SWAHS regional study. Professional capability, competence and effectiveness Considerable confusion continues to exist between key terms like “competence” and “capability”. In this study “competence” is seen as being more related to the possession of the skills and knowledge necessary to perform the duties set down for a particular role; whereas “capability” involves a mixture of emotional and cognitive intelligence including the ability to 1 The professions studied were: accounting, architecture, education, engineering, information technology, journalism, law, nursing, and sports management. Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 28 determine when and when not to deploy these competences. “Capability” cannot be taught, nor can people be trained in it. However, it can be learnt by undertaking educational experiences which entail coming to grips with real world dilemmas (or simulations of them); by experiencing what happens when “things go wrong”, or when the unexpected happens, and then by having to make sense of what is unfolding in order to successfully resolve the situation; and finally to learn from experience by reflecting on what worked or didn’t using a diagnostic framework developed by successful practitioners. Figure 1: Professional Capability Framework Figure 1 summarises the distinction and introduces the key elements of the professional capability framework which, since the UTS studies has been validated in a range of other more extensive investigations, mainly focused on professional leadership (Scott, Coates & Anderson, 2008). In this figure three overlapping aspects of professional capability are identified – personal, interpersonal and cognitive. These domains are underpinned by relevant role-specific and generic competencies (the skills and knowledge found to be essential to the specific role being studied). The research to date has validated a set of items related to each of the five components of Figure 1. The survey used in the SWAHS investigation is the same as that used in these other studies. The items used in the survey fall into a number of scales / domains which align with the domains identified in Figure 1. They are discussed in more detail in Fullan and Scott (2009), Scott, Coates and Anderson (2008), and in Vescio (2005) and are summarised in Appendix 1. In the successful graduate studies those supervising the target group (i.e. the supervisors of graduates who are in their first five years of professional practice) are asked to identify those people they believe are the most effective. They are then asked what criteria/indicators led them to select one graduate as being a particularly effective performer but not another. The successful graduates are asked to rate a set of items on how they personally judge their effectiveness at work. In this way the perception of their supervisors and their own perceptions can be compared. The effectiveness indicators identified in all previous studies have been consolidated and then adjusted with the input of early career nurses and the profession to form the study’s scale “What accounts for your effective performance at work” given in Appendix 1. Personal Capabilities Interpersonal Capabilities Cognitive Capabilities Role-specific Competencies Generic Competencies Capability Competency Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 29 Productive learning in higher education A robust framework and set of checkpoints now exist on what engages students in productive learning and retains them at university. The research that underpins this framework is explored in detail in a commissioned research and analysis report for the 2008 Australian Review of Higher Education (Scott, 2008) and in earlier work (e.g., Foley, 2000; Scott, 2006). This so called RATED CLASS A framework is given in Appendix 2. It is from these empirically verified checkpoints on what is necessary for effective learning that the fourteen items which make up the scale “Keeping university and professional learning relevant” used in the successful graduates studies has been developed (see Appendix 1). The aims of this study were: to identify the criteria by which nursing graduates judge the effectiveness of their own performance at work; to compare these with the indicators used by the Nurse Unit Managers (NUMs) who selected them; then, using the framework in Figure 1 and its associated scales, to identify the capabilities that were seen to be most important for successful nursing practice during the first years following employment as a registered nurse; and to evaluate through “backward mapping”, the degree to which university programs were developing these capabilities and using learning and assessment methods which, in hindsight, the successful graduates had found to have been most productive in helping them prepare for professional practice. These aims were the same as those in the original 2005 UTS study (Rochester, Kilstoff & Scott, 2005) where the same survey was administered to 17 nursing graduates identified as successful by their NUMs at different hospitals. Method Instrument and procedure This study involved graduates from two teaching hospitals in the Western Sydney area. The 2008 Successful Nursing Graduates Survey was delivered by the University of Western Sydney (UWS) Office of Planning and Quality in partnership with the UWS School of Nursing and Midwifery, and SWAHS. The survey was administered to 45 nursing graduates identified by their NUMs as being successful performers in the first five years of their career. The first section of the survey invites participants to rank on a five-point Likert-type scale a range of indicators they would personally use to judge the effectiveness of their performance at work, first on importance (1-low to 5-high) and then on the extent to which they would like to develop these further (1-low to 5-high). In the second section the respondents are asked to rank a range of items that related to the five components of professional capability identified in Figure 1 and Appendix 1 – first on their importance for the successful performance at work (1- low to 5-high) and then, using the same rating scale, on the extent to which the university course and their professional learning experiences have actually focused on each element. The third section of the survey invites participants to rank a range of ways in which their university courses and professional learning programs could be made more relevant to the needs of their profession, first on the relative importance of each strategy to them and then on the extent to which the university and professional learning programs they have experienced have handled that aspect effectively, using the Educational Quality Scale and the same rating system as that outlined above. The survey comprised 64 quantitative items and the results for these items are given in Appendix 1. Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 30 Validity and reliability checks of the capability and context scales to confirm the precision of measurement and the existence of the proposed constructs were conducted while developing a prototype survey instrument in 2008 (Scott, Coates & Anderson, 2008). Statistical methods (e.g., exploratory and confirmatory factor analysis performed in SPSS 2 ) were used to validate the items and scales, and to determine the nature and strength of patterns in response. To facilitate interpretation of quantitative results this paper presents them as a summary of mean based ordinal ranks. A detailed range of qualitative data was also generated. More than 200 comments provided by respondents in the current study were subjected to a thematic qualitative analysis. These comments give important insights into what respondents had in mind when ranking items and identify a wide range of suggestions on what universities and health service providers might now concentrate upon if they want to make their courses relevant to the needs of nursing graduates in their first years of professional work. The project was granted ethics approval by UWS. Results of the 2005 study are shown to allow comparison of the findings. Participants The 45 successful early career nurses who participated in the 2008 Survey were identified through a partnership between senior staff from the UWS School of Nursing and Midwifery and their counterparts in SWAHS. The nurses selected came from two teaching hospitals in an outer-urban area of the city. Respondents had, between them, attended seven universities. Further details of the respondents’ profile are provided in Appendix 3. As noted earlier, selection criteria were not specified to the NUMs who identified the participants. Instead, once they had been selected, each NUM was asked to identify the criteria and indicators they had personally used. This provided an additional data source for the study and, in previous ones, has proven to be of great interest to undergraduate students who are eager to know what their supervisors will be impressed by when they commence work. The participant selection criteria highlighted by NUMs in the 2008 study included: (a) the consistent delivery of allocated work on time and to a specified standard in the time allotted; (b) high levels of client (in this case patient) satisfaction; and (c) high levels of co-worker satisfaction. These indicators align with those identified in earlier studies across the professions and in more recent studies of effective educational leaders (Scott, Coates & Anderson, 2008). Additional, more specific, indicators identified by the NUMs as influencing their choice in the present study included: being motivated and enthusiastic; demonstrated eagerness to learn / advance skills; being pro-active (e.g., initiating talk with a doctor or reviews of patients); sound communication and liaising skills; good organisation and time management skills; and high levels of competency / professionalism. Results The full set of quantitative results is given in Appendix 1. 2 Computer program for statistical analysis, originally Statistical Package for the Social Sciences. Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 31 Indicators used by respondents to judge their own effectiveness at work Table 1 identifies the mean ratings and ranks for the 12 effectiveness indicators respondents could use to judge the quality of their performance at work. These are rated first on importance and then on the extent to which respondents would like to develop further in each performance area (1-low to 5-high). The five highest ranking indicators are highlighted. Table 1: Judging Your Own Effectiveness at Work Importance for work (1 – low to 5 – high) JUDGING YOUR OWN EFFECTIVENESS AT WORK Wish to develop (1 – low to 5 – high) Mean Rank Items Mean Rank 4.27 5 Feeling organised when I do my handover 3.48 11 4.16 9 Successfully handling the emotional needs of the patient. 3.70 5 4.16 8 Helping ensure my patients get the right support when they go home 3.60 8 4.59 2 Successfully finishing the jobs I have to do on time 3.68 7 4.22 6 Receiving positive feedback from my Nurse Unit Manager 3.69 6 4.36 3 Receiving positive feedback from my patients 3.55 9 4.04 10 Receiving positive feedback from colleagues 3.52 10 4.18 7 Achieving goals set for my professional development 3.95 2 4.31 4 Establishing a collegial working environment 3.88 3 4.69 1 Achieving successful patient outcomes 4.10 1 3.40 12 Being invited to discuss my work with others 3.39 12 3.93 11 Successfully implementing new initiatives 3.86 4 The indicators rated high on both importance as a personal effectiveness measure and as a priority for further development include: “Achieving successful patient outcomes” (ranks 1 – on importance, 1 – as a priority for further development); and “Establishing a collegial working environment” (4, 3). The indicators rated high on importance as a personal effectiveness measure but lower as a priority for further development include: “Successfully finishing the jobs I have to do on time” (2, 7); “Receiving positive feedback from my patients” (3, 9); and “Feeling organised when I do my handover” (5, 11). The items rated lower on importance but higher as a priority for further development include: “Achieving goals set for my professional development” (7, 2); “Successfully implementing new initiatives” (11, 4); and “Successfully handling the emotional needs of the patient” (9, 5). Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 32 Capability items ranked highest on importance in 2008 compared with 2005 Table 2 identifies the 12 capability items which attracted the highest rankings on importance for effective performance as an early career nursing graduate (in rank order, highest first). The third column of the table provides the ranks for the relevant items in the 2005 study. Table 2: Capabilities Most Important for Effectiveness at Work WHAT ACCOUNTS FOR YOUR EFFECTIVE PERFORMANCE AT WORK Mean 2008 Rank 2008 Rank 2005 Items 4.75 1 (P) 2 Being willing to face and learn from my errors and listen openly to feedback 4.71 2 (GSK) 3 Being able to organise my work and manage time effectively 4.67 3 (P) 13 Understanding my personal strengths & limitations 4.66 4 (C) 4 Being able to set and justify priorities 4.62 5 (P) 8 Wanting to produce as good a job as possible 4.61 6 (I) 6 Being able to develop and use networks of colleagues to help me solve key workplace problems 4.58 7 (P) 9 Being able to remain calm under pressure or when things go wrong 4.53 8 (C) 7 The ability to use previous experience to figure out what is going on when a current situation takes an unexpected turn 4.53 9 (I) 1 The ability to empathise with and work productively with people from a wide range of backgrounds 4.53 10 (I) 14 Being able to work with senior staff without being intimidated 4.49 11 (P) 11 Having a sense of humour and being able to keep work in perspective 4.48 12 (C) 15 An ability to recognise patterns in a complex situation Professional Capability Framework: P – Personal capabilities scale; I – Interpersonal capabilities scale; C – Cognitive capabilities scale; GSK – generic skills and knowledge scale. Capability items rated high on importance but low on focus at university Items rated high on importance for successful professional practice (mean > 4.0) but rated low on the extent to which the university course focused on this area (mean < 3.0) identify potential areas for enhancement in university and professional learning programs along with their assessment systems. The eight items which meet these criteria are presented in Table 3 along with an indication whether these items had similar gaps between their importance and course focus ratings in 2005. Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 33 Table 3: Capabilities Ranked High on Importance and Lower on University Focus WHAT ACCOUNTS FOR YOUR EFFECTIVE PERFORMANCE AT WORK Mean Importance 2008 Mean University focus 2008 Consistent with 2005 Items 4.58 2.58 (P) yes Being able to remain calm under pressure or when things go wrong 4.53 2.67 (I) yes Being able to work with senior staff without being intimidated 4.20 2.77 (C) no Knowing that there is never a fixed set of steps for solving workplace problems or carrying out a task 4.20 2.84 (I) no Being able to motivate others to achieve great things 4.44 2.86 (I) no Being able to give constructive feedback to work colleagues and others 4.32 2.88 (GSK) no Having a high level of current technical expertise relevant to my work area 4.20 2.93 (C) yes Being able to identify from a mass of detail the core issue in any situation 4.49 2.98 (P) yes Having a sense of humour and being able to keep work in perspective Professional Capability Framework: P – Personal capabilities scale; I – Interpersonal capabilities scale; C – Cognitive capabilities scale; GSK – generic skills and knowledge scale. Capability items rated high on importance and on the focus given to them at university Table 4 shows items that rank high both on importance (mean > 4.0) and university focus (mean > 3.5). These indicate areas of good practice. The choice of a performance rating of 3.5 (i.e. half way between “neutral” and “agree”) is consistent with patterns in other surveys where students consistently rate performance lower than importance. The third column of the table shows whether these items had similar high ratings in the 2005 study results. Table 4: Capabilities Ranked High on Importance and on University Focus WHAT ACCOUNTS FOR YOUR EFFECTIVE PERFORMANCE AT WORK Mean Importance 2008 Mean University focus 2008 Consistent with 2005 Items 4.62 3.77 (P) yes Wanting to produce as good a job as possible 4.33 3.62 (SK) no Being able to manage my own ongoing professional learning and development 4.66 3.58 (I) yes Being able to set and justify priorities 4.53 3.51 (IP) no The ability to empathise with and work productively with people from a wide range of backgrounds Professional Capability Framework: P – Personal capabilities scale; I – Interpersonal capabilities scale; C – Cognitive capabilities scale; GSK – generic skills and knowledge scale. Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 34 When one’s capability as a nurse is most tested After completing the capability sections of the survey respondents were invited to indicate under what circumstances their professional capability as a nurse was being most tested. The majority of respondents (71.1%) reported that it is “when things are going wrong”, 15.6% reported that it is “when things are going normally” and only 1 respondent out of the 45 surveyed reported that this happens “when things are going well”. Strategies for making university and professional learning relevant: Importance and performance ratings in 2008 Table 5 identifies the mean ratings and ranks for the 14 strategies found to make university courses and professional learning programs in Nursing more interesting, engaging, relevant and productive. Each strategy is first rated on importance in keeping university and professional learning relevant and then on the extent to which the university course or professional learning programs which respondents have undertaken so far has handled that aspect effectively. The five highest ranking methods are highlighted. Table 5: Strategies for Making University and Professional Learning Relevant: Mean Ratings and Ranks Importance for work KEEPING UNIVERSITY AND PROFESSIONAL LEARNING RELEVANT University’s focus Mean Rank Items Mean Rank 4.38 8 Focus more directly on the capabilities identified by successful nurses as being important in university courses and assessment 3.03 12 4.49 6 Use real-life workplace problems identified by successful graduates as a key resource for learning 3.43 6 4.38 9 Make work-placements which test out the capabilities identified as most important in this study a key focus in each course 3.40 7 4.78 1 Clinical practice 3.81 2 4.58 3 Rotation through a range of clinical areas 3.76 3 4.33 10 Use successful graduates more consistently as a learning resource in as a learning resource in university courses 2.83 14 2.96 14 Decrease the amount of formal classroom teaching of basic technical skills and use self-instructional guides and I.T. to develop these 2.86 13 4.60 2 Include learning experiences based on real-life case studies that specifically develop the interpersonal and personal skills needed as a nurse 3.69 4 4.46 7 Provision of a Transition to Practice Program (if you have undertaken one) 3.82 1 3.69 13 When relevant, use I.T. to make learning as convenient and interactive as possible 3.27 11 4.24 11 Ensure that all teaching staff model the key attributes identified as being important in this study 3.38 9 4.58 4 Ensure that teaching staff have current workplace experience 3.31 10 4.56 5 Make assessment more real-world and problem-based and less focused on memorising factual material 3.39 8 4.20 12 Use performance on the capabilities identified as being most important in earlier parts of this survey as the focus for assessment and feedback on all learning tasks 3.44 5 Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 35 The strategies rated high on both importance and the extent of their current use in university and professional learning program include: “Clinical practice” (ranks: 1, 2); “Include learning experiences based on real-life case studies that specifically develop the interpersonal and personal skills needed as a nurse” (2, 4); and “Rotation through a range of clinical areas” (3, 3). These items indicate perceived areas of good practice in the university programs undertaken by respondents. It is important to note that twelve of the items attract mean importance ratings well in excess of 4.0. This validates the areas covered by these twelve items as being important Quality Assurance checkpoints for both the design of learning programs for nurses and for tracking the effectiveness of their implementation. The qualitative data generated by the survey and summarised in the next section confirms this. The items rated high on importance but relatively low on the extent to which universities or professional learning programs handle them effectively included: “Ensure that teaching staff have current workplace experience” (ranks 4, 10); and “Make assessment more real-world and problem-based and less focused on memorising factual material” (5, 8). Qualitative comments A wide range of qualitative data was generated by the survey. First, as already noted, respondents were asked to explain their high ratings for each of the quantitative areas surveyed (“best aspect” comments) and to suggest improvements (“needs improvement” comments). They were also asked to identify one of the most challenging situations which they have encountered in their early years of professional practice and how they went about addressing it. Their responses provide ideal material with which to develop real-world scenarios for case and problem-based learning and assessment. The challenges identified in the 2008 survey included: having to manage a wide range of emergencies like a patient suddenly becoming unstable; having to cope with a wide range of demands bearing down at the same time; having to manage when a number of colleagues are unexpectedly ill or called away; dealing with “difficult” patients and distressed families; and having to work productively with a diverse range of staff and senior people. Extensive qualitative feedback was provided when respondents were asked how, in hindsight, their university and professional learning might have been made more relevant. Consistent with the highest rating items in Table 5, in their open-ended feedback respondents give particular attention to clinical practice. Further, many participants’ comments about the importance of clinical practice suggested that this practice should be continuous, self directed and that it should operate within a structured environment which provides ongoing opportunities for interaction with patients. Other “best aspect” learning methods recurrently mentioned in the comments all involve active learning and include: laboratory practicals; lecturers’ real life stories; discussions of trouble shooting issues; simulations of case scenarios; and Registered Nurses discussing their experiences and sharing effective ways to deal with common challenges. Many respondents identified specific course subjects like the following as proving to be relevant in the early years of professional practice: Anatomy; Physiology; Pathophysiology; Pharmacology / Medication; Patient Assessment; Nursing Care; and Ethics. Taken more generally what respondents in this survey say will improve their professional learning aligns closely with the RATED CLASS A framework and confirmed in the commissioned research and analysis report on student learning to the 2008 Review of Australian Higher Education (Scott, 2009). Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 36 Discussion and conclusions This study of 45 successful early career nursing graduates from two outer-urban Sydney hospitals has both tested a methodology and generated a range of findings about professional effectiveness, capability and learning in the unique operating context of being a nurse. In terms of professional effectiveness it should be noted that ten of the 12 effectiveness indicators attracted a mean importance rating 4.0 or higher, suggesting that the vast majority of respondents saw them as being either moderately high or very high in their usefulness. The highest ranking indicators align closely with the indicators identified independently by the NUMs who chose the sample and with the effectiveness indicators identified by supervisors in the earlier studies of other professions. In terms of the capabilities ranked highest on importance there is a considerable overlap between the 2008 and 2005 results. Some variation is seen, however, in capability areas like the following which appear in the top 12 items ranked highest on importance in the 2008 study but not in the 2005 study: “Understanding my personal strengths and limitations”; “Being able to work with senior staff without being intimidated”; and “An ability to recognise patterns in a complex situation”; along with the following item which received the top rank in 2005, but was ranked ninth in 2008: “The ability to empathise with and work productively with people from a wide range of backgrounds”. In both the 2005 and 2008 studies eight out of the twelve items ranked highest on importance relate to emotional intelligence (i.e. some aspect of personal or interpersonal capability). This pattern of findings aligns with the studies undertaken in other professions and those involving professional leadership (Fullan & Scott, 2009; Scott, Coates & Anderson, 2008; Vescio, 2005). Specifically, in 2008 five of the top ranking items come from the Personal Abilities scale and 3 from the Interpersonal Abilities scale. A further three items relate to the Cognitive Ability scale. Just one item in the top twelve comes from the generic skills and knowledge scale. There is a similar pattern in the results of the 2005 study – with the Personal Abilities scale again producing the largest number of high rating capability items. Of eight capabilities ranked high on importance but low on focus given at university four align with the 2005 Survey results and may warrant improvement attention in nurse education programs. They are: “Being able to remain calm under pressure or when things go wrong”; “Being able to work with senior staff without being intimidated”; “Being able to identify from a mass of detail the core issue in any situation”; and “Having a sense of humour and being able to keep work in perspective”. Of particular interest is the relatively low rating given to the existing university focus on the item: “Having a high level of current technical expertise relevant to my work area”. The “needs improvement” qualitative data generated by the survey can be used to supplement these results and explain the ratings in more detail. The following areas were consistently identified in both the 2008 and 2005 Survey results as being well addressed in the university programs undertaken by respondents: “Wanting to produce as good a job as possible”; and “Being able to set and justify priorities”. With a scale up of the study to the state or national level it will become increasingly possible to identify areas of good practice in addressing high rating capability items in one location that can then be shared with another. The “best aspect” qualitative data generated by the survey can be used to supplement these results. It has been said that the emotional intelligence capabilities identified as being so important to effective early career performance in this profession cannot be taught. This is true but they can be learnt, especially if the supervisors of clinical practice are alerted to the top rating emotional intelligence and cognitive capabilities, and if their assessment of trainee nurses gives focus to how well the individual displays them when things go wrong or when the Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 37 unexpected happens. This was supported by an earlier study of successful Engineering graduates (Scott & Yates, 2002) and further discussed in the commissioned report to the 2008 Review of Australian Higher Education (Scott, 2009). The focus on clinical practice highlighted by the successful graduates as a positive aspect of university program also aligns with more extensive analyses of some 280,000 qualitative “best aspect” and “needs improvement” comments made on the national Course Experience Questionnaire at the field of education level (Scott, 2006; Scott, Grebennikov & Gozzard, forthcoming) where the analysis of preferred methods in the health field of education shows that almost 90% of the “best aspect” comments are about clinical practice. It should be noted however that, in these broader studies and in the current one, there are suggestions for making the clinical experience far more focused on the emotional intelligence and cognitive capabilities found to be so important in all these successful graduate studies. Some ways in which this might be done have been identified earlier in this paper. The learning strategies rated high on importance but relatively low on the extent to which universities or professional learning programs handle them effectively included: “Ensure that teaching staff have current workplace experience”; and “Make assessment more real-world and problem-based and less focused on memorising factual material”. These items identify potential areas for improvement action in undergraduate nurse education and professional learning programs. The “needs improvement” qualitative data generated by the survey can be used to explain the quantitative results and to identify relevant improvement solutions. With a scale up of the study to the state or national level it will again become increasingly possible to identify areas of good practice in addressing high rating items on the Educational Quality Scale in one location that can then be shared with another. The “best aspect” qualitative data generated by the survey can be used to supplement these results. The study confirms that both capability and competence are important elements in being identified as a successful graduate. It shows that key skills and knowledge are necessary but are not sufficient for capable and consistently effective practice. The study’s findings align with those from all previous investigations. The study shows, for example, that capability as an early career nurse is most tested when the unexpected happens, when many small challenges all unfold at the same time and when things go “wrong”. It reveals that emotional intelligence – both personal and interpersonal – is a core ingredient in framing an effective response, along with a distinctive diagnostic and contingent way of thinking. It shows that key aspects of emotional intelligence and the cognitive skills required cannot be taught but they can certainly be learnt – especially if clinical practice, simulations and case-based learning experiences are appropriately designed and the capabilities that rate highest on importance to successful graduates are given central focus. The study also confirms the importance of bringing theory, practice and real life situations into dialogue whenever possible and of seeing content as being just one element in the process of “reading and matching” (Scott, 1999; Schön, 1983, Fullan and Scott, 2009). This also suggests that greater use of integrated and problem-based assessment focused on the highest rating early career capabilities identified in the study may be warranted. The study identifies a range of effectiveness indicators that can be used to assess the quality of early career performance. A relatively small sample size and possible self-reporting bias are two limitations of the study which may have impacted the reliability of the outcomes. Nonetheless, studies like this can assist directly with the process of repositioning higher education. And, if replicated across professions, sites and systems, they represent a distinctive and significant way to help governments assure the standards, relevance and cost-benefit of university programs suited to the emerging needs of the 21 st century. Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 38 References Business Council of Australia (2009). Fit for the job: Adapting to Australia’s new healthcare challenges (Report). BCA, Melbourne. Accessed on 22 August 2010 at: http://www.bca.com.au/Content/101545.aspx Department of Education, Employment and Workplace Relations (2008). Review of Australian Higher Education: Final Report. DEEWR: Canberra. Dewey, J. (1933). How we think. Lexington, Massachusetts: Heath & Co. Elmore, R. F. (1979). Backward mapping: Implementation research and policy decisions. Political Science Quarterly, 94(4), 601-616. Foley, G. (2000). Understanding adult education and training (2nd Edition). Sydney: Allen & Unwin. Fullan, M., & Scott, G. (2009). Turnaround leadership for higher education. San Francisco: Jossey Bass. Gillard, J. (2009). Speech at the Universities Australia Conference. Canberra: ACT. Accessed on 22 August 2010 at: http://www.deewr.gov.au/Ministers/Gillard/Media/Speeches/Pages/Article_090304_1557 21.aspx Goleman, D. (1998). Working with emotional intelligence. London: Bloomsbury. Gonczi, A., Hager, P. & Oliver, L. & Australia. Dept. of Employment, Education and Training. & Australia. National Office of Overseas Skills Recognition. (1990). Establishing competency based standards for the professions. (Research Paper No. 1, NOOSR). Canberra: Australian Government Publishing Service. Rochester, S., Kilstoff, K., & Scott, G. (2005). Learning from success: Improving undergraduate education by understanding the capabilities of successful nurse graduates. Nurse Education Today, 25(3), 181-188. Schön, D. (1983). The reflective practitioner. New York: Basic Books. Scott, G. (1999). Change matters: Making a difference in education and training. London: Unwin. Scott, G. (2006). Accessing the student voice (Report). DEEWR, Canberra. Accessed at: http://www.dest.gov.au/sectors/higher_education/publications_resources/profiles/access _student_voice.htm on 22 August 2010. Scott, G (2008): Student Engagement and Satisfaction with Learning, (Commissioned research and analysis report to the Australian Review of Higher Education). DEEWR, Canberra. Accessed on 22 August 2010 at: http://www.deewr.gov.au/HigherEducation/Review/Documents/Scott.pdf Scott, G., Coates, H. & Anderson, M. (2008). Learning Leaders in Times of Change. Sydney: Australians Learning and Teaching Council. Scott, G., Grebennikov, L. & Gozzard, T. (2009). ICT-enabled learning: the student perspective. Journal of Institutional Research, 14(2), 1-16. Scott, G. & Saunders, S. (1995). The continuous learning improvement program for Australia’s skill Olympians. Sydney: Work Skill Australia Foundation Incorporated. Scott, G. & Yates, W. (2002). Using successful graduates to improve the quality of undergraduate engineering programs. European Journal of Engineering Education, 27(4), 363-378. Tough, A. (1979). The Adult’s Learning Projects: A Fresh Approach to Theory and Practice in Adult Learning (2nd ed.). Toronto: Ontario Institute for Studies in Education. Vescio, J. (2005). An investigation of successful graduates in the early stages of their career across a wide range of professions (Final Report). University of Technology, Sydney. http://www.bca.com.au/Content/101545.aspx http://www.deewr.gov.au/Ministers/Gillard/Media/Speeches/Pages/Article_090304_155721.aspx http://www.deewr.gov.au/Ministers/Gillard/Media/Speeches/Pages/Article_090304_155721.aspx http://www.dest.gov.au/sectors/higher_education/publications_resources/profiles/access_student_voice.htm http://www.dest.gov.au/sectors/higher_education/publications_resources/profiles/access_student_voice.htm http://www.deewr.gov.au/HigherEducation/Review/Documents/Scott.pdf Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 39 Appendix 1: Quantitative results Successful Graduate Capability Study, Nursing 2008-09 (N = 45) Five highest ranks within the first and last sections are highlighted green. Ten highest ranked abilities are highlighted yellow. Importance for work Items Wish to develop Mean Rank JUDGING YOUR OWN EFFECTIVENESS AT WORK Mean Rank 4.27 5 1. Feeling organised when I do my handover 3.48 11 4.16 9 2. Successfully handling the emotional needs of the patient. 3.70 5 4.16 8 3. Helping ensure my patients get the right support when they go home 3.60 8 4.59 2 4. Successfully finishing the jobs I have to do on time 3.68 7 4.22 6 5. Receiving positive feedback from my Nurse Unit Managers 3.69 6 4.36 3 6. Receiving positive feedback from my patients 3.55 9 4.04 10 7. Receiving positive feedback from colleagues 3.52 10 4.18 7 8. Achieving goals set for my professional development 3.95 2 4.31 4 9. Establishing a collegial working environment 3.88 3 4.69 1 10. Achieving successful patient outcomes 4.10 1 3.40 12 11. Being invited to discuss my work with others 3.39 12 3.93 11 12. Successfully implementing new initiatives 3.86 4 WHAT ACCOUNTS FOR YOUR EFFECTIVE PERFORMANCE AT WORK Importance for work PERSONAL ABILITIES Extent Uni focused Mean Rank Mean Rank 4.75 1 1. Being willing to face and learn from my errors and listen openly to feedback 3.47 7 4.67 3 2. Understanding my personal strengths & limitations 3.20 15 3.73 36 3. Being confident to take calculated risks and take on new projects 2.72 35 4.58 7 4. Being able to remain calm under pressure or when things go wrong 2.58 37 4.29 20 5. Having the ability to defer judgement and not to jump in too quickly to resolve a problem 3.13 18 4.29 21 6. A willingness to persevere when things are not working out as anticipated 3.00 25 4.62 5 7. Wanting to produce as good a job as possible 3.77 1 4.00 32 8. Being willing to take responsibility for projects, including how they turn out 3.07 22 4.22 24 9. Having an ability to make a hard decision 3.13 19 4.40 16 10. A willingness to pitch in and undertake menial tasks when needed 3.30 11 4.49 11 11. Having a sense of humour and being able to keep work in perspective 2.98 28 Importance for work INTERPERSONAL ABILITIES Extent Uni focused Mean Rank Mean Rank 4.53 9 1. The ability to empathise with and work productively with people from a wide range of backgrounds 3.50 5 4.45 13 2. A willingness to listen to different points of view before coming to a decision 3.44 8 4.62 6 3. Being able to develop and use networks of colleagues to help me solve key workplace problems 3.40 9 4.36 17 4. Understanding how the different groups that make up my organisation operate and how much influence they have in different situations 3.00 24 4.53 10 5. Being able to work with senior staff without being intimidated 2.67 36 4.44 14 6. Being able to give constructive feedback to work colleagues and others 2.86 31 4.20 27 7. Being able to motivate others to achieve great things 2.84 32 4.27 22 8. Being able to develop and contribute positively to team-based projects 3.33 10 Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 40 Importance for work INTELLECTUAL ABILITIES Extent Uni focused Mean Rank Mean Rank 4.20 25 1. Knowing that there is never a fixed set of steps for solving workplace problems or carrying out a task 2.77 34 4.20 26 2. Being able to identify from a mass of detail the core issue in any situation 2.93 29 4.53 8 3. The ability to use previous experience to figure out what is going on when a current situation takes an unexpected turn 3.14 17 4.05 30 4. Being able to diagnose what is really causing a problem and then to test this out in action 3.00 27 4.25 23 5. An ability to trace out and assess the consequences of alternate courses of action and, from this, pick the one most suitable 3.07 21 4.43 15 6. Being able to readjust a plan of action in light of what happens as it is implemented 3.11 20 4.16 28 7. Being able to see how apparently unconnected activities are linked and make up an overall picture 3.21 14 4.66 4 8. Being able to set and justify priorities 3.58 4 4.48 12 9. An ability to recognise patterns in a complex situation 3.00 23 Importance for work SPECIFIC SKILLS AND KNOWLEDGE Extent Uni focused Mean Rank Mean Rank 4.32 19 1. Having a high level of current technical expertise relevant to my work area 2.88 30 3.76 35 2. Being able to use I.T. effectively to communicate & perform key work 3.25 13 4.33 18 3. Being able to manage my own ongoing professional learning and development 3.62 3 3.49 38 4. An ability to chair and participate constructively in meetings 2.56 38 3.84 34 5. Being able to make effective presentations to clients/colleagues 3.70 2 4.04 31 6. Understanding the role of risk management and litigation in current professional work 3.47 6 3.62 37 7. Knowing how to manage projects into successful implementation 3.20 16 4.14 29 8. An ability to help others learn in the workplace 3.00 26 3.91 33 9. Understanding how organisations like my current one operate 2.84 33 4.71 2 10. Being able to organise my work and manage time effectively 3.27 12 Importance KEEPING UNIVERSITY AND PROFESSIONAL LEARNING RELEVANT Extent of use Mean Rank Mean Rank 4.38 8 1. Focus more directly on the capabilities identified by successful nurses as being important in university courses and assessment 3.03 12 4.49 6 2. Use real-life workplace problems identified by successful graduates as a key resource for learning 3.43 6 4.38 9 3. Make work-placements which test out the capabilities identified as most important in this study a key focus in each course 3.40 7 4.78 1 4. Clinical Practice 3.81 2 4.58 3 5. Rotation through a range of Clinical areas 3.76 3 4.33 10 6. Use successful graduates more consistently as a learning resource in as a learning resource in university courses 2.83 14 2.96 14 7. Decrease the amount of formal classroom teaching of basic technical skills and use self-instructional guides and I.T to develop these 2.86 13 4.60 2 8. Include learning experiences based on real-life case studies that specifically develop the interpersonal and personal skills needed as a nurse 3.69 4 4.46 7 9. Provision of a Transition to Practice Program (if you have undertaken one) 3.82 1 3.69 13 10. When relevant, use I.T. to make learning as convenient and interactive as possible 3.27 11 4.24 11 11. Ensure that all teaching staff model the key attributes identified as being important in this study 3.38 9 4.58 4 12. Ensure that teaching staff have current workplace experience 3.31 10 4.56 5 13. Make assessment more real-world and problem-based and less focused on memorising factual material 3.39 8 4.20 12 14. Use performance on the capabilities identified as being most important in earlier parts of this survey as the focus for assessment and feedback on all learning tasks 3.44 5 Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 41 Appendix 2 RATED CLASS A Framework R Are immediately RELEVANT to their particular background, abilities, needs and experiences; A Provide more opportunities for ACTIVE learning than they do for passive learning – in particular when they include frequent opportunities for students to make contact with people who are further down the same learning path and to actively search a range of relevant data bases; T Consistently link THEORY WITH PRACTICE – for example, when they consistently use case and problem based learning, carefully structured clinical practice and relevant simulations; E Effectively manage students' EXPECTATIONS right from the outset; D Ensure that learning proceeds logically, is integrated and that the course of study has clear DIRECTION; C Use a valid graduate CAPABILITY profile to specifically generate appropriate outcomes and assessment tasks. Care is taken to ensure that the units of study used directly address the full capability profile and do not duplicate each other; and that all learning feeds assessment; L Provide them with opportunities to pursue flexible LEARNING PATHWAYS. Although students are allowed some flexibility and choice in the study units undertaken, careful attention is given to ensuring that they still end up with the same spread and quality of capabilities at graduation; A Ensure that ASSESSMENT tasks are valid, integrated, problem-based and that feedback on them is timely, constructive and detailed; S Not only include opportunities for SELF-MANAGED LEARNING using both digital and paper-based resources but actively coach students on how to undertake it; S Provide SUPPORT AND ADMINISTRATIVE SERVICES which are easily accessed, responsive to students needs and which specifically work together to optimise the total experience which a student has of the university or college; A Ensure that ACCESS to learning times, locations and resources makes participation in the learning program as convenient and productive as possible. Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 42 Appendix 3 Profile of respondents to the 2008 study Current job title Frequency Percent Acting Clinical Nurse Educator 3 6.7 Acting Nurse Unit Manager 1 2.2 Clinical Nurse Specialist 1 2.2 Nursing Unit Manager 1 2.2 Registered Nurse 35 77.8 Registered Nurse Anaesthetics 1 2.2 Registered Nurse Intensive Care 2 4.4 Registered Nurse Medications 1 2.2 Total 45 100.0 Main activities undertaken in present job Frequency Percent Acute geriatric nursing 1 0.9 Assist patient activities of daily living (ADLs) 2 1.8 Attend high risk deliveries 1 0.9 Basic nursing care 3 2.7 Bed managing 1 0.9 Cardiac monitoring 4 3.5 Care for very premature babies 1 0.9 Chemotherapy 1 0.9 Communication with doctors 2 1.8 Documentation 2 1.8 Family support 1 0.9 Fluid / airway / pain management 8 7.1 Hygiene 1 0.9 Medication Administration 19 16.8 Nurse caring for sick children 1 0.9 Patient advocacy 5 4.4 Patient assessment / re-assessment 7 6.2 Patient education 4 3.5 Patient observation 5 4.4 Post-op care 7 6.2 Pre-op care 3 2.7 Research nurse 1 0.9 Rosters 2 1.8 Student / new staff support and education 8 7.1 Team work / leader 9 8.0 Total / direct patient care 14 12.4 Total 113 100.0 Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 43 Main degree relevant to present employment Frequency Percent Bachelor of Nursing 40 88.9 Bachelor of Nursing and PG Certificate 1 2.2 Bachelor of Nursing, Grad Certificate in Trauma and Orthopaedics 1 2.2 Bachelor of Science in Nursing 2 4.4 Diploma in Adult Nursing 1 2.2 Total 45 100.0 University at which this degree was completed Frequency Percent Australian Catholic University 8 17.8 Charles Sturt University 3 6.7 University of Manchester 1 2.2 University of Newcastle 1 2.2 University of Sydney 6 13.3 UTS 5 11.1 University of Western Sydney 21 46.7 Total 45 100.0 Year this degree was completed Frequency Percent 2003 4 8.9 2004 9 20.0 2005 18 40.0 2006 10 22.2 2007 3 6.7 Unspecified 1 2.2 Total 45 100.0 Being employed as an Assistant in Nursing whilst undertaking the nursing degree Frequency Percent Yes 29 60.0 No 17 37.8 Total 45 100.0 Scott, G., Chang, E. & Grebennikov, L. (2010). Using successful graduates to improve the quality of undergraduate nursing programs. Journal of Teaching and Learning for Graduate Employability, 1(1), 26 – 44. 44 Sector of employment as an Assistant in Nursing Frequency Percent Aged Care 2 6.9 Aged Care / casual pools Westmead and Nepean private 1 3.4 Aged Care, Private Hospital 1 3.4 Agency, private sector, nursing home 1 3.4 Both Public and Private 1 3.4 Casual Pool 2 6.9 Central Sydney Area Health 1 3.4 Emergency Surgical 1 3.4 Enrolled Nurse 3 10.3 Hospital 1 3.4 Hospital as a casual employee 1 3.4 Intensive care as a nurses aid and specialling on all other wards 1 3.4 Nursing Home 4 13.8 Nursing Home and Hospital 1 3.4 Orthopaedic, spinal and neurological rehabilitation 1 3.4 Private 1 3.4 Private Nursing Home 1 3.4 Public Hospital 2 6.9 Public Hospital and Nursing Nome 1 3.4 Stockton 1 3.4 UG Westmead Pools 1 3.4 Total 29 100.0