Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 2 Delivering work-ready business graduates – keeping our promises and evaluating our performance Denise Jackson 1 , Ruth Sibson 1 and Linda Riebe 1 , d.jackson@ecu.edu.au; r.sibson@ecu.edu.au; l.riebe@ecu.edu.au 1 Edith Cowan University Abstract Business schools globally are responding to calls for graduate work-readiness primarily through the development of employability skills, encompassing career management skills, and work integrated-learning (WIL). There has been considerable attention to clarifying precisely which skills should be developed, and how, but far less on evaluating employability skill provision and its impact on graduate work-readiness. This is increasingly important as industry worldwide continues to lament graduate inadequacies in certain employability skills and the extent to which they are job-ready. This paper outlines a systematic approach for evaluating employability skill outcomes and the effectiveness of learning programs in developing these skills. The approach was developed and is currently being implemented in a learning program dedicated to developing employability skills in business undergraduates in an Australian university. It may assist other universities in communicating, assessing, mapping and reporting their employability skills outcomes; an integral component of all business undergraduate programs, and now a requirement of all Australian higher education providers (TEQSA, 2011). The approach provides a means of evaluating program effectiveness in skill provision, enabling a more informed review of curricula content, assessment and pedagogical techniques to achieve better alignment with industry requirements. Keywords: employability, graduate, undergraduate, business, competencies, generic skills. Background Business schools worldwide are investing significant effort and resources to respond to industry calls for work-ready business graduates. Work-readiness is typically addressed in two ways in business undergraduate programs: non-technical skill development, encompassing career management skills, and the incorporation of Work-integrated Learning (WIL) or internship opportunities. The benefits of undertaking a period of learning in a workplace setting as part of one’s degree program are well documented and focus on enhanced technical and non -technical skill outcomes, and graduate ability to transfer acquired skills and knowledge to the workplace (see Freudenberg, Brimble & Cameron, 2011). Different manifestations of WIL exist globally, each highly regarded by industry. The UK ‘sandwich’ degree program, where students complete 2 years of university study, followed by one year in industry and a final year of university study, has long been considered superior to standard degree programs in regards to developing graduate work-readiness (Confederation of British Industry [CBI], 2009). Similarly, US internship programs are also considered to prepare students better for the workplace, in addition to enhancing their employment prospects (Gault, Leach & Duey, file:///C:/Users/258909K/Desktop/d.jackson@ecu.edu.au file:///C:/Users/258909K/Desktop/r.sibson@ecu.edu.au mailto:l.riebe@ecu.edu.au Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 3 2010). A review of recent literature reveals a growing abundance of studies examining the positive impact of student work placements on graduate work-readiness. Non-technical skills, often referred to as generic, professional or key skills and herein defined as employability skills, are those which facilitate the creative and productive application of disciplinary skills and knowledge in the workplace. Team working, communication, critical thinking and problem-solving are prominent examples of employability skills considered vital in business graduates (Jackson & Chapman, 2012). Employability skills are widely considered as a pivotal and integral element of business undergraduate education (CBI, 2009) and equally as important as disciplinary content (Hancock et al., 2009). They are firmly entrenched in relevant professional association accreditation criteria, such as America’s Association to Advance Collegiate Schools of Business [AACSB]; EQUIS, operated by the European Foundation for Management Development; CPA Australia; and the UK’s Quality Assurance Agency for Higher Education (QAA). They also feature heavily in the recently developed Learning and Teaching Academic Standards for Australian undergraduate degree programs (Australian Learning and Teaching Council [ALTC], 2010), further underscoring the importance of work-readiness in business graduates. These standards prescribe that all graduating students must have a ‘systematic and coherent body of knowledge, principles and concepts and higher order learning skills for further learning and professional employment’ (Australian Qualifications Framework Council [AQFC], 2009, p. 13). Acknowledging the need to produce graduates who can add immediate value in job roles, there is a wealth of international research dedicated to clarifying industry-relevant employability skills in undergraduates (Business Council of Australia [BCA], 2006; CBI, 2009; Council for Industry and Higher Education [CIHE], 2008; National Leadership Council for Liberal Education & America’s Promise [LEAP], 2008).There has also been considerable research on pedagogical strategies for successfully developing these skills; with universities either opting for bolt-on programs or embedding employability skill outcomes into core curricula. Cranmer (2006), for example, provides an overview of these two different approaches in her review of how academics are engaging with the teaching and learning of employability skills. Unfortunately, there is not equal rigour in evaluating the success of employability skill development in business undergraduate programs; with the use of simple measures, such as graduate employment statistics, not effectively capturing and accurately reflecting the quality of graduates (Barrie, Andrews, Dean & Heimanis, 2009). This does not detract from exemplars of good practice in articulating graduate capabilities, such as the use of student and course portfolios at Curtin University in Western Australia (Oliver, 2011), but highlights the need for a university-wide, systematic approach in evaluating employability skill outcomes and the degree to which such programs or approaches are meeting employers’ needs. This is amplified further by evidence of gaps between expectations of graduates and business school provision of certain employability skills, and also by rising employer expectations (Hart Research Associates, 2010). Gaps are particularly evident in communication (Kotzee & Johnston, 2011), problem solving, critical thinking and leadership skills (CIHE, 2008); all vital elements of the managerial skill set. As industry laments the lack of work-readiness in business graduates it is important to acknowledge this is not the sole responsibility of the business school. Graduate work- readiness, equating here to the degree to which a graduate is employable, has several determinants: their ability to transfer acquired skills (Rae, 2007), workplace awareness (Bennett, Dunne & Carre, 1999) and job mobility (Wittekind, Raeder & Grote, 2010). Employability skill outcomes form only one aspect of employability and are influenced by Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 4 external factors such as graduates’ exposure to activities and relationships beyond education and work (Wheeler, 2008) and skill development during their school years (Smith & Green, 2005). Despite these influences beyond their control, business schools should be actively evaluating their success in achieving employability skill outcomes to identify ways of improving graduate work-readiness. Reasons for ambivalence in assessing employability skill outcomes and the quality of provision by business schools may be attributed to the absence of appropriate skill frameworks for measuring these skills. It may also be due to a lack of funding and required resources dedicated to evaluation purposes, further aggravated by increasing administrative workloads in academic staff (Tight, 2010). Some academics may challenge the need for higher education outcomes to align with, and respond to, industry requirements; arguing this detracts from other important goals such as nurturing societal good and developing intellect in students (Starkey & Tempest, 2009). The benefits of evaluating employability skill provision apply to all stakeholders in undergraduate education. Educators can better gauge their contribution to graduate work- readiness and are alerted to areas of weakness requiring adjustments in pedagogical strategies, program content, learning activities and/or assessment. Graduates will be better equipped to explicitly promote their own capabilities in employability skills, which is particularly important given their prominence in recruitment selection criteria. Finally, evaluation outcomes will highlight those areas of learning programs which would benefit from industry assistance and intervention, such as embedding professional learning activities into curricula and assessment. This paper aims to outline a systematic approach for evaluating employability skill outcomes and the effectiveness of learning programs in developing these skills. It may assist other schools and faculties in the vital process of evaluating their outcomes and provision, and would apply to skill development programs using either a bolt-on or embedded approach. The context for the adopted methodology is a program, core to the university’s undergraduate business degree, designed to explicitly engage students in developing employability skills using a student-centred, experiential teaching and learning approach in an environment closely emulating the workplace. It comprises four sequential units: two in first year, one in second year, and one in third year. The program aims to be at the forefron t of responding to the voice of employers in developing graduates sufficiently equipped to add immediate value in the workplace. The outlined methodology for evaluating employability skill provision was catalysed by the need to articulate clearly defined program outcomes and evidence of the effectiveness of the program to key internal and external stakeholders. The proposed approach for evaluating skill outcomes and learning program effectiveness is currently being implemented in the said program; an outline and evaluation of each phase of the approach is presented. Methodology for evaluating employability skill provision Figure 1 (see below) summarises the steps adopted in evaluating the development of employability skills. An explanation of each step is provided with a broad rationale from relevant literature and details on specific actions completed within the program. Phase one: Project set-up First, a project team was established to determine the aims and parameters of the evaluative research strategy. These aims were effectively communicated and justified to achieve the required buy-in from relevant academic staff. Given the high ratio of casual academic staff in Australian Business schools (Sciulli, Smith & Ross, 2009), a budgetary allowance for their input into skill development issues and assistance in data gathering may be required. The Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 5 aims and parameters for the program’s team were (i) to develop a contemporary and relevant contextualised employability skills framework which delineates the program’s key messages for stakeholders by clearly establishing the aims and scope of the program, and (ii) to evaluate the program’s success in achieving the targeted skills. Being able to articulate key messages explicitly for the overall program required a shift from the silo development of skills to an overall program development of skills through a mapping process, necessitating ‘buy in’ from all unit coordinators. Figure 1: Evaluating Employability Skill Outcomes Phase one: Project set-up Phase two: Develop and communicate a contextualised employability skills framework Phase four: Audit student skills against framework Phase three: Map curriculum against skills framework Phase five: Review and re-align curriculum and assessment Phase six: Evaluate transfer of skills from classroom to workplace External environment Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 6 Phase two: Develop and communicate a contextualised employability skills framework A strategic and methodological approach to the development, assessment and reporting of employability skills in a university context first requires a coherent Employability Skills Framework (ESF). This framework should capture and reflect current employer preferences, be unambiguous in meaning and measurable in both learning and work environments. It should also be contextualised to reflect the history, goals and requirements of individual institutions, faculties and/or learning programs (Taylor, 2005; Hampson & Junor, 2009). The development of a contextualised ESF is, arguably, the most vital step in the process of evaluating employability skill outcomes in the university setting. The role of ESFs contextualised to individual institutions, faculties or programs is multi- faceted. First, they provide an overarching strategic direction on addressing employability skills by structuring government, academic and employer perspectives on what constitutes graduate employability (Dacre Pool & Sewell, 2007). This is invaluable, given the somewhat overwhelming mass of international literature in this area. Jackson (2009) discusses the importance of systematically profiling industry requirements of business graduates. An ESF is one example of a profile of targeted skills in graduates. Second, they communicate to stakeholders precisely what employability skills are being addressed by a particular institution/faculty/program (Dacre Pool & Sewell, 2007; Rae, 2007). This may impact upon the marketability of certain degree programs. Third, they inform and guide curricula content and may clarify suitable pedagogical practices for developing the defined employability skills. Finally, they provide a means of mapping and benchmarking university/faculty/program efforts in achieving graduate employability (Yorke & Knight, 2004). Frameworks provide a tool for monitoring student competency in the defined skills through regular peer, self and lecturer performance reviews. Realising the multi- faceted role of a contextualised ESF does, however, require institutional commitment. A number of replicable steps defined the program’s approach to developing its own skills framework. Determine the starting point. In developing the program’s ESF, Australia’s national ESF was determined as the logical starting point. The national ESF framework, developed by The Australian Chamber of Commerce and Industry [ACCI] and the BCA for the Department of Education, Science and Training [DEST], 2002, comprises skills and attributes deemed critical for new and existing employees across a broad range of Australian work contexts. The skills consist of communication, teamwork, problem solving, self-management, planning and organising, technology, lifelong learning and initiative and enterprise. The attributes comprise loyalty, commitment, honesty and integrity, enthusiasm, reliability, personal presentation, common sense, positive self-esteem, sense of humour, balanced attitude to work and home, ability to deal with pressure, motivation and adaptability. Although individual job roles demand different skill levels and priorities, the framework is considered generic across different industries. Barrie (2005) challenges the parallel treatment of skills and attributes. He argues that attributes concern personal traits whereas skills can be acquired through practice. They combine, however, to form an overarching capability which determines a graduate’s ability to function in the workplace (Jackson & Chapman, 2012). The national framework has been central to the vocational education and training (VET) sector with providers adopting a system wide approach to incorporate the eight employability skills across their entire range of national training packages. Despite the key role Australian Higher Education Providers (HEP) also play in the economic, social and cultural development of the nation, their response has been more haphazard. Their focus has been Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 7 institution-specific and frequently differs across different faculties with no commonly agreed framework for employability skill development (BIHECC, 2007). The national framework is considered a subset of the extensive research on graduate attributes (BIHECC, 2007), the latter catalysed by the West Review (DETYA 1998) which synthesised a set of generic attributes all graduates should have. Although BIHECC’s (2007) elementary review of practices indicated that graduate attribute development across HEPs addressed the ESF, some (Barrie, 2005; Freeman, Hancock, Simpson, Sykes, Petocz, Densten & Gibson, 2008) believe not all Australian universities are progressing beyond a myriad of recommended policy achievement statements and surface mapping strategies, and that a more systematic approach to developing and reporting employability skills is required. Such concerns are also heightened by certain aspects of HEP funding which depend on the development and reporting of graduate attributes (Barrie, 2006). Identify and improve on the starting point’s weaknesses. There are a number of identified problems inherent in the Australian ESF. Weaknesses include, first, the separation of attributes from skills; described as ‘political defeat’ (Hampson & Junor, 2009, p. 8) for those employers who expressed what they needed in graduates to the BCA and ACCI. Initially it was acknowledged that the attributes ‘contribute to overall employability’ (ACCI, 2002 p. 4) and would be included in the framework, yet they were later removed; most likely due to difficulties in developing and measuring them (Taylor, 2005; Hampson & Junor). The incorporation of these attributes into the framework would facilitate more rigorous mapping and benchmarking processes, in addition to acknowledging the importance of developing certain ‘attitudinal dispositions and affective traits’ sought by employers (Taylor, p. 205). Second, the meanings of individual skills are poorly defined and ambiguous; a problem plaguing many studies on required generic skills in graduates (Male & Chapman, 2005). The precise meaning of the eight skills is open to interpretation and confirms disparity in academics’ understanding of the exact nature of employability skills (Taylor, 2005; Bridgestock, 2009). Such ambiguity causes inevitable problems in assessment and measurement. For example, how is one to measure: ‘having a personal vision and goals’? Further refinement of the attributes and skill elements is required. Third, Hampson and Junor (2009) argue the national framework fails to consider the level of skill required in different workplace contexts. They advocate differentiating between routine and complex tasks and assessing employability skills at basic, creditable and advanced status. Fourth, they argue many elements of the ESF reflect work processes/activities (such as ‘negotiating responsively’), rather than the actual skills needed to perform these activities (such as ‘being able to defend and assert one’s rights, interests and needs coherently and convincingly’). Fifth, they maintain many of the cited skills require workplace experience; less problematic in higher education given the majority of Australian undergraduates undertake part-time employment during their degree studies (McInnis & Hartley, 2009). Finally, it is important the framework captures and represents the needs and priorities of today’s employers. Certainly for the VET sector there were concerns over a lack of consultation with industry representatives on the skills and attributes selected for the ESF (Down, 2003). In the face of recent economic volatility and changing social and environmental pressures, it is pertinent that skill requirements are regularly reviewed to ensure currency. The changing face of the management profession from trends, such as flattening organisational structures virtual working and diverse labour composition, will impact specifically on the skills critical in business graduates. The ESF should remain current and be continually aligned with stakeholder expectations through tracking research studies, industry engagement and networking with other educational practitioners. Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 8 Incorporate current literature. Current literature should guide the development of a contextualised ESF beyond the starting point frame of reference. More recently, for example, Jackson and Chapman (2012) produced a competency framework (see Table 1) that identifies forty-five workplace behaviours from an extensive review of international, employer-based studies on employability skills dating back to 1998 (Jackson, 2010). The behaviours are subsequently grouped into twenty competencies (treated in the same way as skills), and are based on empirical and conceptual associations identified in the literature review. Table 1: Framework of Competencies and Constituent Behaviours, adapted from Jackson and Chapman (2012) Competency Behaviour C1: Business principles Use of business concepts C2: Core business skills Numeracy Technology C3: Critical thinking Pattern recognition and conceptualisation Evaluation C4: Problem solving Analytical/convergent reasoning Diagnosing C5: Decision management Lateral thinking/creativity Information management Decision making C6: Political skills Influencing others Conflict resolution C7: Working with others Task collaboration Team working Social intelligence Cultural and diversity management C8: Oral communication Verbal communication Giving and receiving feedback C9: Personal ethics Personal ethics C10: Confidence Self-efficacy C11: Self-awareness Meta-cognition Lifelong learning C12: Self-discipline Self-regulation Stress tolerance Work/life balance C13: Innovation Entrepreneurship Change management C14: Leadership Project management Performance management Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 9 Meeting management Developing others C15: Formal communication skills Public speaking Meeting participation Written communication C16: Performance Efficiency Multi-tasking Autonomy C17: Organisational skills Goal and task management Time management C18: Environmental awareness Organisational awareness Commercial awareness C19: Professional responsibility Social responsibility Accountability C20: Work ethic Drive Initiative Significantly, Jackson and Chapman’s (2012) framework seeks to overcome a number of the issues in the Australian ESF. First, the competencies encompass and do not discriminate between skills and attributes. Second, and arguably the overarching feature, is that the competencies, and their constituent behaviours, are far more clearly defined, and therefore more assessable, than their national predecessors. Taking the example of ‘Communication’, more measureable behaviours such as ‘give and receive feedback appropriately and constructively’ and ‘speak publicly and adjust their style according to the nature of the audience’ have replaced the somewhat ambiguous elements from the Australian ESF of ‘sharing information’ and ‘speaking clearly and directly’. These behaviours represent processes or activities which more clearly define competence for a given employability skill. A shared understanding of the targeted skills, among employers, students and academics, is essential for student learning (see Price, O’Donovan, Rust & Carroll, 2008). Third, this use of processes and activities define more precisely the behaviours constituting the employability skills, and better indicate the extent to which each must be performed effectively in the workplace. For example, ‘develop a range of solutions using lateral and creative thinking’ defines the required outcome and the attributes and/or capabilities for successful performance. In combination, this increases the likelihood of strong content validity in instruments used to develop and assess the identified behaviours, thus increasing the chances of their successful development. Hampson and Junor (2009) acknowledge the difficulties in defining required levels of skills in different contexts within the VET sector. There is varying opinion on whether skill requirements for the undergraduate cohort are more homogenous; some maintaining they do not vary greatly by discipline (Billings, 2003) others advocating the significant influence of disciplinary context (Jones, 2010). Fourth, their framework is considered current as it derives from an extensive and timely review of literature on industry-relevant graduate skills, and it incorporates a number of elements under-represented in the Australian ESF. For example, leadership skills, considered critical for graduate employability (Hancock et al., 2009), are incorporated through a collection of skills comprising a ‘leadership’ competency and a heavy focus on the ‘softer’ skills of self-regulation and self-awareness. These are widely perceived as constituting Emotional Intelligence and fundamental to effective leadership (Boyatzis, 2009). Also vital to graduate employability is critical thinking (Papadopoulos, 2010), addressed Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 10 through behaviours evaluating detail and conceptualising the ‘bigger picture’. This focus on ‘helicopter vision’ continues with greater attention to understanding and evaluating organisational, national and global environments and social responsibility, the latter considered key in the wake of the global financial crisis. Notably, the elements of risk management and mastery of a second language have been omitted from Jackson and Chapman’s (2012) framework, both deemed beyond the scope of employability skills and more relevant to disciplinary content. Certain attributes, namely ‘loyalty’, a ‘sense of humour’, ‘personal presentation’ and ‘common sense’, were also not directly included as these were considered either to be unnecessary or addressed elsewhere. Develop the finalised framework. The resulting framework comprises ten overarching skills with 40 constituent behaviours and their related descriptors (see Table 2). There are a number of differences between Jackson & Chapman’s (2012) framework and the contextualised one. First, it comprises ten employability skills, as opposed to twenty competencies. The identification and use of ten employability skills was considered more manageable for mapping and benchmarking purposes and more consistent with the individual unit’s learning outcomes which were based initially on the national ESF. Where appropriate, the new overarching skills were written as activities or processes to encourage cognitive awareness of the active learning process and to reduce ambiguity. Constituent behaviours of the original competencies were merged into their new, host skill area with the exception of behaviours from the ‘Work Ethic’ and ‘Decision Management’ competencies which were dispersed across a number of the employability skills. Second, the behaviours ‘use of business concepts’ and ‘commercial awareness’ were removed due to their disciplinary focus and coverage in other core units within the undergraduate business degree program. The technical behaviours comprising the leadership competency were also removed on the premise that some students might not realistically be able to develop these prior to graduation given they prescribe supervisory workplace experience. Third, five behaviour names were changed, each marked by one asterisk (*), to reduce potential misinterpretation, more accurately reflect the level of skill development addressed in the program and/or reflect more contemporary terminology recently adopted in the employability skills arena. Fourth, an expanded definition of multi- tasking was given to avoid potential misinterpretation, and a slight amendment to the definition of ‘self-efficacy’ and four instances of the word ‘understand’ were replaced with more measurable phrases within the behaviour descriptors, all marked with two asterisks (**). Finally, the program’s ESF incorporates some additional skill elements. First, in light of growing international concerns for our environment, the concept of sustainability was affirmed through minor amendments to the social responsibility and lifelong learning behaviour descriptors. In addition, ‘career management’ was inserted as an additional behaviour under self-awareness. Bridgestock (2009) reviews literature on the positive relationship between career management and employability, concluding it plays a pivotal role in enhancing economic growth. In the face of continued global economic volatility and increasingly competitive graduate labour markets (Tomlinson, 2008); the need for developing career management skills in graduates is being increasingly recognised (Popovic & Thomas, 2009). The emphasis of this new behaviour is on developing informed career goals and understanding local labour market conditions and career opportunities which, in combination with appraising one’s own strengths and weaknesses – termed ‘meta-cognition’ in the program’s framework, will enhance employability (Eby, Butts & Lockwood, 2003). Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 11 Table 2: Contextualised Employability Skills Framework Unit 1 Unit 2 Unit 3 Unit 4 T P A T P A T P A T P A W o rk in g e ff e c ti v e ly w it h o th e rs Task collaboration Complete group tasks through collaborative communication, problem solving, discussion and planning. Team working Operate within, and contribute to, a respectful, supportive and cooperative group climate. Social intelligence Acknowledge the complex emotions and viewpoints of others and respond sensitively and appropriately**. Cultural and diversity awareness* Work productively with people from diverse cultures, races, ages, gender, religions and lifestyles. Influencing others Defend and assert their rights, interests and needs and convince others of the validity of one’s point of view. Conflict resolution Address and resolve contentious issues with key stakeholders. C o m m u n ic a ti n g e ff e c ti v e ly Verbal communication Communicate orally in a clear and sensitive manner which is appropriately varied according to different audiences and seniority levels. Giving and receiving feedback Give and receive feedback appropriately and constructively. Public speaking Speak publicly and adjust their style according to the nature of the audience. Meeting participation Participate constructively in meetings. Written communication Present knowledge, in a range of written formats, in a professional, structured and clear manner. S e lf - a w a re n e s s Meta-cognition Reflect on and evaluate personal practices, strengths and weaknesses in the workplace. Life-long learning Actively seek, monitor and manage knowledge and sustainable opportunities for learning in the context of employment and life. Career management Developing meaningful and realistic career goals and pathways for achieving them in light of labour market conditions. Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 12 T h in k in g c ri ti c a ll y Conceptualisation* Recognise patterns in detailed documents and scenarios to understand the ‘bigger’ picture. Evaluation Recognise, evaluate and retain key points in a range of documents and scenarios. A n a ly s in g d a ta a n d u s in g te c h n o lo g y Numeracy Analyse and use numbers and data accurately and manipulate into relevant information. Technology Select and use appropriate technology to address diverse tasks and problems. Information management Retrieve, interpret, evaluate and interactively use information in a range of different formats. P ro b le m - s o lv in g Reasoning* Use rational and logical reasoning to deduce appropriate and well- reasoned conclusions. Analysing and diagnosing* Analyse facts and circumstances and ask the right questions to diagnose problems. Decision making Make appropriate and timely decisions, in light of available information, in sensitive and complex situations. D e v e lo p in g in it ia ti v e a n d e n te rp ri s e Entrepreneurship/ Intrapreneurship* Initiate change and add value by embracing new ideas and showing ingenuity and creativity in addressing challenges and problems. Lateral thinking / creativity Develop a range of solutions using lateral and creative thinking. Initiative Take action unprompted to achieve agreed goals. Change management Manage change and demonstrate flexibility in their approach to all aspects of work. S e lf - m a n a g e m e n t Self-efficacy Be self-confident in dealing with the challenges that employment and life present**. Stress tolerance Persevere and retain effectiveness under pressure or when things go wrong. Work / life balance Demonstrate the importance of well being and strive to maintain a productive balance of work and life**. Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 13 Self-regulation Reflect on and regulate their emotions and demonstrate self- control**. S o c ia l re s p o n s ib il it y Social responsibility Behave in a manner which is sustainable and socially responsible (e.g., consistent with company policy and/or broader community values). Accountability Accept responsibility for own decisions, actions and work outcomes. Personal ethics Remain consistently committed to and guided by core values and beliefs such as honesty and integrity. Organisational awareness Recognise organisational structure, operations, culture and systems and adapt their behaviour and attitudes accordingly**. D e v e lo p in g p ro fe s s io n a li s m Efficiency Achieve prescribed goals and outcomes in a timely and resourceful manner. Multi-tasking Perform more than one task at the same time**. Autonomy Complete tasks in a self-directed manner in the absence of supervision. Time management Manage their time to achieve agreed goals. Drive Go beyond the call of duty by pitching in, including undertaking menial tasks, as required by the business. Goal and task management Set, maintain and consistently act upon achievable goals, prioritised tasks, plans and realistic schedules. Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 14 In addition to those addressed by Jackson and Chapman’s (2012) framework, and thus incorporated into the program’s ESF, further weaknesses with the Australian framework were overcome. First, it is based entirely on measurable workplace task processes and activities. This overcomes the conceptual confusion associated with the mixing of task processes and skills, and is appropriate to the program given its ethos of focusing on developing employability skills in an environment which emulates the workplace. Second, and to overcome the problem of developing skills which require workplace experience, the program creates precisely the environment in which to successfully learn these defined behaviours and therefore does not preclude those who do not have suitable work experience. Third, the development of behaviours considered beyond the scope of the program, such as technical leadership skills, were removed. The ESF also provides further assurance of currency, thus capturing and reflecting the needs of contemporary employers, through incorporating additional behaviours identified by academics as pertinent to the program. To ensure compliance with university-wide policy, the ESF was also cross referenced in a matrix table (adapted from Rumsey, 2005), against the national framework to ensure skills were sufficiently covered. Following this mapping exercise, the university’s own targeted graduate attributes were cross referenced to ensure inclusion. Communicate the framework. The framework was communicated to stakeholders in a number of different ways. Students on the employability skills program were introduced to the framework through its inclusion in Unit Plans, class posters which were actively referenced during class sessions and postcards for individual students. The framework was also broadcast electronically, via intranet and wikis, across the university as an exemplar of initiatives in graduate employability. It constitutes what Price et al. (2008, p.177) describe as an appropriate forum ‘for the development and sharing of standards within and between disciplinary and professional communities’. Promotional literature for external stakeholders incorporated the framework to delineate the program’s mission and key outcomes. Phase three: Map curriculum against framework A curriculum map of employability skill development within the program was completed. This was undertaken for a number of reasons. First, to delineate key messages for both internal and external stakeholders of the benefits of skill development within a program offering; second, to ascertain where the skill is explicitly taught, practiced and/or assessed over the life of the program; third, to identify and update the core skills, and their constituent behaviours, distinctly associated with each unit within the program and, finally, to revise unit learning outcomes, content and descriptions to explicitly promote the core skills developed in each unit. Literature acknowledges the considerable uses and benefits of curriculum mapping for industry educators and students themselves (Cox & King, 2006), and as a valuable platform to promote a supportive and collaborative culture among colleagues (Uchiyama & Radin, 2008). A standard mapping process may be extrapolated to different types of employability skill offerings. The project team first established the mapping criteria which is dependent on the size, rigour and, most likely, funding for the exercise. Coordinators mapped each skill, and its associated behaviours, against learning activities, opportunities for practice and assessments to ensure units are constructively aligned (see Biggs, 2006). This gave a rich picture of how each behaviour is developed within each unit and, therefore, across the learning program. It facilitated linear mapping for each skill, and constituent behaviours, to create a picture of how they are scaffolded as students proceed through the program. A less arduous exercise would have been to assign core skills to units within the learning program and map how these are taught, practised and assessed. While this evaluates the Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 15 overall achievement of learning outcomes across the program, it provides little information on whether skill acquisition is sequentially scaffolded as students’ progress through the program, important for student learning (see Meyers & Nulty, 2009). Also, as the learning environment allows for the integration of many employability skills across learning activities and assessment items, the ability to effectively silo core skills to individual units is questionable. Next, a mapping model was determined. There are a range of curricula mapping models developed for course alignment and articulation (see Tariq, Scott, Cochrane, Lee & Ryles, 2004). Models to map employability skills include matrix models (Yorke & Knight, 2004); computer software models developed by universities to map entire degree programs (Oliver, Ferns, Whelan & Lilly, 2010); template designs (Fallows & Steven, 2000) and concept models (Harden, 2001). Robley, Whittle and Murdoch-Eaton (2005) combined four maps to show declared, developed, learned and assessed data to demonstrate an ‘alignment loop’. Bath, Smith, Stein and Swann (2004) question whether mapping is enough to ensure curricula is aligned with what students actually learn and propose a cycle of planning (mapping the intended curriculum), acting (embedding), reviewing (comparing teacher and student experiences) and reflecting (redesigning the curriculum). For the program, mapping parameters were initially defined as unit learning outcomes (ULOs) against employability skills agreed by the project team as core to that particular unit. In accordance with the Curtin University computer software model (Oliver et al., 2010), it was also agreed that staff would identify the specific university-level graduate attribute and the level of thinking according to Bloom’s revised taxonomy (Krathwohl, 2002) associated with each ULO. The model, however, was designed to map a smaller number of university specific graduate attributes and there were no budgetary allowances within the program for adapting their computer model to the ten skills within the ESF. Further, the mapping of content against ULOs can be problematic as the alignment of ULOs with delivered and assessed content is not always guaranteed (McNeill, Gospel & Hedberg, 2008). It was agreed, therefore, that ULOs should be verified, and possibly amended, as an outcome of mapping rather than act as a mapping parameter. This exercise is a way of ensuring that learning outcomes are aligned with curricula content and assessment. Thus, a matrix model was chosen to simplify the mapping process. It provided a visual representation of skills taught (T), practised (P) and assessed (A) across the program and was easy to complete using associated guidelines for staff. See Tariq, Scott, Cochrane & Ryles (2004) for further examples of the use of matrix models. The matrix model comprised a spreadsheet posted in an electronic community wiki, mapping each of the 40 behaviours against what was taught, practised and assessed in their particular unit. These were collated into a colour-coded curriculum map which indicated T, P and A for each behaviour across the four units. A linear review of each skill set, and its constituent behaviours, was also undertaken to assess, according to the TPA data, how it is sequentially developed as students progress through the program. Phase four: Audit student skills against framework Upon completing a unit, students within the program undertake a Skills Audit in an online survey environment. They reflect on their expected level of performance in the workplace for each of the defined behaviours and how, and how well, a particular unit develops them. This provides rich data for evaluating unit/program effectiveness and identifying themes in skill development to complement existing curricula mapping exercises. Encouraging students to self-reflect on their progress in developing the defined skills may improve their learning and understanding of the skills (Boud & Garrick, 1999). Students must be supported in this self - reflection process to stimulate a critical thinking approach as the act of self -reflection does Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 16 not always result in learning (Lizzio & Wilson, 2007). Students may keep a longitudinal journal, as part of their skills portfolio, recording observations of incidents/events where they have encountered employability skills and provide supporting evidence of use on which they can then critically reflect; a form of critical incident analysis (Tripp, 1993). Over time, a catalogue of evidence to support student understanding and usage of each skill would be constructed. This ongoing construction of understanding through high quality reflection would aid transfer of skills to new situations as students become more adept at explicitly identifying employability skills. In addition to engendering learning, student self-assessment against devised skill frameworks will benefit them in a number of ways. First, the skills audit clarifies and provides evidence of individual performance in different skill areas, enhanced by available e-portfolio platforms. It will better equip candidates to effectively develop career plans, curriculum vitae and address selection criteria. Second, it is essential for students to ‘buy-in’ to the importance of employability skills, for the purposes of their own work-readiness and for universities to meet the demands of today’s employers. Careful consideration of the extent of their mastery of different skills, and precisely how this was achieved, will engage students better with the meaning and importance of graduate work-readiness. Finally, there is considerable literature on the difficulties of achieving the successful transfer of skills from the university classroom to the workplace (see Leberman, McDonald & Doyle, 2006). Reflecting on one’s learning – to better understand what has been learned and why - improves transfer across different learning and application contexts (Bransford & Schwartz, 1999). From an institutional perspective, students reflecting on and self-rating their abilities in the defined skills may provide rich data for evaluating unit and/or program effectiveness in skills development. A continual cycle of skill auditing provides longitudinal data which tracks individual and group performance in employability skill development over an extended period of time, enabling the fine-tuning of pedagogical practices and curricula content to improve learning program effectiveness. In addition, the students build a comprehensive skills audit which maps their employability skills against learning and assessment activities as they progress through the program. This form of student self-reporting complements and enriches existing curriculum mapping evidence on program strengths and weaknesses in developing certain skills. Graduate self-reporting, however, is a subjective measure of performance (Halfhill & Nielsen, 2007) and there is a greater chance of intentional and unintentional measurement error although Allen and Van der Velden (2005) discuss solutions for validating and testing self-assessment instruments to reduce this problem. Resources such as SPARK (Freeman & McKenzie, 2002) evaluate undergraduate performance through self, peer and facilitator assessments and, for those students completing WIL or internship programs, employer assessments. This 360 degree approach may provide a more useful evaluation of student performance and is something the program hopes to introduce in the near future. Phase five: Review and re-align curriculum and assessment Phase five requires a review of the mapping and audit outcomes to identify program areas which are excelling and others requiring improvement; this phase has not yet been undertaken within the program. Combining objective and subjective measures, such as mapping and student self-reported data, should provide a rich evaluation of program performance. Areas identified as performing poorly will be reviewed in regard to pedagogical strategies, opportunities for practice, assessment and content. Conversely, areas performing well will be cited as examples of best practice to inform others. The framework will inform the revision and development of more aligned curriculum content and learning and assessment activities. Price et al. (2008) emphasise the importance of Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 17 valid assessment which addresses intended learning outcomes, that being targeted skills defined by the framework. They also note the importance of dialogue between academics and students on assessment standards. This is facilitated by the ESF which explicitly provides an instrument for discussion, understanding and feedback – among all stakeholders - of a set of expected outcomes in business graduates. The use of explicit assessment standards, such as the behaviour descriptors, may improve consistency and objectivity in marking yet Sadler (2009) argues students being bound by ‘tightly specified criteria’ (p. 178) may be detrimental to their learning. As units continuously evolve, it is important to track the impact of changes in content or assessment on a curricula map. Those co-ordinating units within the relevant curriculum must report changes, and their impact on the map recorded, or the entire mapping process must be revisited regularly. A process for reporting changes to teaching, student practice and/or assessment should be implemented. Phase six: Evaluate transfer of skills from classroom to workplace Phase six in operationalising an ESF requires the measurement of the transfer or conversion of acquired skills from the classroom to the workplace (see Figure 1). This represents the move from measuring employability skill provision to measuring the impact on graduate work readiness and their actual performance in the workplace. Again, this part of the model has not yet been implemented although the planned approach is to track graduates of the program into the workplace and continue to audit their skill portfolio and compare with others from different universities. Transfer is an extremely grey and challenging area of educational theory and practice, the responsibility of which lies with HEPs, graduate employers and the undergraduates themselves (Eraut, 2004). Each stakeholder must contribute to creating a healthy environment for transfer by capitalising on the process’ potential facilitators and managing likely inhibitors. Gathering data on graduate workplace performance in the defined skills, in addition to their performance as undergraduates in the classroom, will facilitate measuring the degree to which learning is transferred across acquisition and application contexts. This final phase is critically important yet often overlooked (Hakel & Halpern, 2005). Encouraging academics and undergraduates to focus on transfer is considered to have a positive impact on the transfer process. This may be initiated by incorporating an additional behaviour, termed ‘application of knowledge’ and defined as: ‘translate theory into practice by successfully adjusting and applying technical knowledge in a range of scenarios’, into the framework. Associated problems and lessons learned It is important to note that this model has not yet been rigorously evaluated but rather represents an evidence-informed approach. Evaluation, and further refinement of the model, is therefore warranted. It is hoped, however, that documenting this approach to evaluating employability skill outcomes and provision may assist other universities in their quest to systematically develop, assess and report on employability skills, moving beyond poorly constructed ESFs and surface mapping strategies. There have been challenges associated with the process depicted in Figure 1 and a number of lessons have been learnt from the program’s implementation of the model to date. First, the model is a continuous one which requires institutional support and considerable commitment from academic staff. Second, the skills framework must be reviewed regularly to address issues with construct validity. A common concern is that behaviours are transparent in meaning, can be easily measured, and capture emerging research on Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 18 changes in employer requirements. This is particularly important in this extended period of economic uncertainty and flux. Already there is an argument for incorporating an additional behaviour termed research; defined as ‘developing an awareness of the meaning and importance of knowledge and how it is created, reviewed and advanced in an ethical and systematic manner’. This reflects the broad opinion that research-based learning considerably enhances graduate employability (Jenkins, 2009). On a more conceptual level, some may argue the process is misguided as an assessment of competence in higher education does not provide an assurance of work readiness (Down, 2003). Developing the skills defined in the framework in undergraduates does not guarantee their successful application in graduate positions in the workplace. Conclusion This defined approach hopes to assist universities in declaring, assessing, mapping and reporting employability skills outcomes; now formally requirement by HEPs (Tertiary Education Quality and Standards Agency [TEQSA], 2011). A systematic approach to evaluating employability skill outcomes and provision in higher education first requires a valid framework of the skills, knowledge and attributes required in graduates. The Australian Employability Skills Framework (DEST, 2002) was designed for a broad range of cohorts from schools, the VET sector and higher education. This, and certain design issues, urges the development of more contemporary frameworks that are contextualised to individual university/faculty/program needs. The proposed framework addresses the problems of the national one and provides a sound foundation for evaluating the provision of employability skills in a university setting. Notwithstanding the challenges of implementing the framework, there are significant benefits which may be translated across different learning programs and faculties. From a Faculty perspective, the framework can delineate and communicate to stakeholders (particularly students, their parents, academics and employers) the program mission, aims and outcomes. This could improve undergraduate program marketability and choice, as well as reduce the silo unit culture. The framework can also be used to map current curricula offerings to evaluate program outcomes and the extent to which it meets industry needs. It may provide assurance of learning for local employers and relevant accrediting bodies. It also has the potential to highlight areas requiring review, which is critical given the impact of prevailing employability skill ‘gaps’ in graduates, and exemplars of good practice for the wider community. It will also assist with devising scaffolding pathways for sequential learning and avoids duplication of skill development across the different units. For graduates, the proposed approach encompasses a skills audit through self -reporting. This documents their capabilities against a comprehensive skills framework considered vital to today’s employers. The process of self-reflection will assist them in addressing selection criteria, as well as highlight the importance of employability skills and any areas requiring improvement; one seen as vital as undergraduates acknowledge the importance of actively addressing employability skill development (Nilsson, 2010). Transparent evaluations of employability skill provision will also inform employers of aspects of the curricula which may benefit from their input, such as incorporating professional learning activities (see Lawson, Fallshaw, Papadopoulos, Taylor & Zanko, 2011). Auditing and mapping outcomes combine to provide a rich picture of a program’s effectiveness in addressing skills defined in the framework. This will inform a more systematic review and development of the program’s content, learning outcomes, pedagogical techniques, learning activities and assessment. It will also streamline units, and the program as a whole, allowing better alignment with the defined framework and, therefore, industry requirements. Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 19 References Allen, J. & Van Der Valden, R. (2005). The role of self-assessment in measuring skills. Maastricht, Netherlands: Research Centre for Education and the Labour Market. Australian Chamber of Commerce and Industry [ACCI] (2002). Employability skills - An employer perspective: Getting what employers want out of the too hard basket. Canberra: Australian Government Publishing Service. Australian Learning and Teaching Council [ALTC] (2010). The learning and teaching academic standards project – final report. Strawberry Hills, NSW: ALTC. Australian Qualification Framework Council [AQFC] (2009). Strengthening the AQF: An architecture for Australia’s qualifications. Adelaide: AQFC. Barrie, S. (2005). Rethinking Generic Graduate Attributes. HERDSA News, 1 April 2005. Barrie, S. (2006). Understanding what we mean by the generic attributes of graduates. Higher Education, 51(2), 215–241. Barrie, S., Andrews, J., Dean, L. & Heimanis, I. (2010). Employability: Realising the potential of a university education. In S. Denton & S. Brown (Eds.) A practical guide to university and college management: Beyond bureaucracy. London: Routledge, 283– 302. Bath, D., Smith, C., Stein, S. & Swann, R. (2004). Beyond mapping and embedding graduate attributes: Bringing together quality assurance and action learning to create a validated and living curriculum. Higher Education Research & Development, 23(3), 313–328. Bennett, N., Dunne, E. & Carre, C. (1999). Patterns of core and generic skill provision in Higher Education. Higher Education, 37(1), 71–93. Biggs, J. & Tang, C. (2006). Teaching for quality learning at university: What the student does. Buckinghamshire: Society for Research into Higher Education and Open University Press. Boud, D. & Garrick, J. (1999). Understanding learning at work. London: Routledge. Bransford, J. & Schwartz, D. (1999). Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education, 24(1), 61-100. Business Council Australia [BCA] (2006) Changing paradigms: Rethinking innovation policies, practices and programs. BCA, Melbourne. Business, Industry and Higher Education Collaboration Council [BIHECC] (2007). Graduate employability skills. Canberra: BIHECC. Boyatzis, R. (2009). Competencies as a behavioural approach to emotional intelligence. Journal of Management Development, 28(9), 749–770. Bridgestock, R. (2009). The graduate attributes we’ve overlooked: Enhancing graduate employability through career management skills. Higher Education Research and Development, 28(1), 31–44. Confederation of British Industry [CBI] (2009). Future fit: Preparing graduates for the world of work. London. Council for Industry and Higher Education [CIHE] (2008). Graduate employability: What do employers think and want? London: CIHE. Cox, S. & King, D. (2006). Skill sets: An approach to embed employability in course design. Education + Training 48(4), 262–274. Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 20 Cranmer, S. (2006). Enhancing graduate employability: best intentions and mixed outcomes.Studies in Higher Education, 31(2), 169–84. Dacre Pool, L. & Sewell, P. (2007). The key to employability: Developing a practical model of graduate employability. Education + Training, 49(4), 277–289. Department of Education, Science and Training [DEST] (2002). Employability skills for the future. Canberra: DEST. Department of Education, Training and Youth Affairs [DETYA] (1998). Learning for life: review of higher education financing and policy. Report for Higher Education Financing and Policy Reviewing Committee (West Review). Canberra: DETYA. Down, C. (2003). Employability Skills: Revisiting the Key Competencies or a new way forward? Proceedings of the11th Annual International Conference on Post- Compulsory Education and Training, Queensland, Australia, Centre for Learning Research, Griffith University,171–178. Eby, L., Butts, M. & Lockwood, A. (2003). Predictors of success in the era of the boundaryless career. Journal of Organizational Behavior, 24, 689–708. Eraut, M. (2004). Transfer of knowledge between education and workplace settings. In H. Rainbird, A. Fuller & A. Munro (Eds.) Workplace learning in context. London: Routledge, 201–221. Freeman, M. & McKenzie, J. (2002). SPARK, a confidential web–based template for self and peer assessment of student teamwork: benefits of evaluating across different subjects. British Journal of Educational Technology, 33(5), 551–569. Freeman, M., Hancock, P., Simpson, L., Sykes, C., Petocz, P., Densten, I. & Gibson, K., (2008). Business as usual: a collaborative and inclusive investigation of existing resources, strengths, gaps and challenges to be addressed or sustainability in teaching and learning in Australian university business faculties. Accessed by http://www.altc.edu.au/project-business-usual-collaborative-sydney-2006. Freudenberg, B., Brimble, M. & Cameron, C. (2011). WIL and generic skill development: The development of business students’ generic skills through work-integrated learning. Asia-Pacific Journal of Cooperative Education, 12(2), 79–93. Gault, J., Leach, E. & Duey, M. (2010). Effects of business internships on job marketability: the employers’ perspective. Education + Training, 52(1), 76–88. Hakel, M. & Halpern, D. (2005). How far can transfer go: Making transfer happen across physical, temporal and conceptual space. In J. Mestre (Ed.) Transfer of learning from a modern multidisciplinary perspective (357–370). Greenwich, CT: Information Age Publishing. Halfhill, T. & Nielsen, T. (2007). Quantifying the ‘softer side’ of management education: An example using teamwork competencies. Journal of Management Education, 31(1), 64–80. Hampson, I. & Junor, A. (2009). ‘Employability’ and the substance of soft skills. Proceedings of the 27th International Labour Process Conference, Edinburgh, Scotland. Hancock, P., Howieson, B., Kavanagh, M., Kent, J., Tempone, I. & Segal, N. (2009). Accounting for the future: More than numbers, Volume 1. Australian Learning and Teaching Council, Strawberry Hills, NSW: ALTC. Hart Research Associates (2010). Raising the bar: Employers’ views on college learning in the wake of the economic downturn. Washington, DC: Association of American Colleges and Universities. Jackson, D. (2009). Profiling industry relevant management graduate competencies: The need for a fresh approach. International Journal of Management Education, 8(1), 85– 98. http://www.altc.edu.au/project-business-usual-collaborative-sydney-2006 Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 21 Jackson, D. (2010). An international profile of industry relevant competencies and skill gaps in modern graduates. International Journal of Management Education, 8(3), 29–58. Jackson, D. & Chapman, E. (2012). Non-technical competencies in undergraduate Business degree programs: Australian and UK perspectives. Studies in Higher Education, 37(5), 541–567. Jenkins, A. (2009). Research-teaching linkages: Enhancing graduate attributes. Overview: The aims, achievements and challenges from the Enhancement Theme. Report for Quality Assurance Agency [QAA]. UK: QAA. Jones, A. (2010). Generic attributes in Accounting: The significance of disciplinary context. Accounting Education, 19(1-2), 5–21. Kotzee, B. & Johnston, R. (2011). ‘Can’t string a sentence together’?. UK employers’ views of graduates’ writing skills. Industry and Higher Education, 25(1), 45–52. Krathwohl, D. (2002). A revision of Bloom's taxonomy: An overview. Theory into Practice, 41(4), 212–218. Lawson, R., Fallshaw, E., Papadopoulos, T., Taylor, T. & Zanko, M. (2011). Professional learning in the Business curriculum: Engaging industry, academics and students. Asian Social Science, 7(4), 61–68. Lieberman, S., McDonald, L. & Doyle, S. (2006). The transfer of learning: Participants' perspectives of adult education and training. Aldershot, UK: Gower Publishing. Liberal Education & America’s Promise [LEAP] (2007). College Learning for the New Global Century. Washington: The National Leadership Council for LEAP. Lizzio, A. & Wilson, K. (2007). Developing critical professional judgement: The efficacy of a self-managed reflective process. Studies in Continuing Education, 29(3), 277–293. Male, S. & Chapman, E. (2005). Assessing the generic competencies of Engineering graduates: Preliminary report from an ongoing research project. Proceedings of the 2005 ASEE / AAEE 4 th Global Colloquim on Engineering Education. McInnis, C. & Hartley, R. (2009). Managing study and work: The impact of full-time study and paid work on the undergraduate experience in Australian universities. Report for the Department of Education, Science and Training [DEST]. Canberra: DEST. McNeill, M., Gosper, M. & Hedberg, J. (2008). Engaging students with higher order learning (or not): Insights into academic practice. Proceedings of the ATN Assessment Conference 2008, 20–21 November, University of South Australia, Adelaide. Meyers, N. & Nulty, D. (2009). How to use (five) curriculum design principles to align authentic learning environments, assessment, students’ approaches to thinking and learning outcomes. Assessment & Evaluation in Higher Education, 34(5), 565–577. Nilsson, S. (2010). Enhancing individual employability: The perspective of engineering graduates. Education + Training, 52(6/7), 540–551. Oliver, B. (2011). Assuring graduate capabilities: An approach to determining and evidencing standards. Proceedings of the Australian Quality Forum 2011, Melbourne, Australia, 120–126. Oliver, B., Ferns, S., Whelan, B. & Lilly, L. (2010). Mapping the curriculum for quality enhancement: Refining a tool and processes for the purpose of curriculum renewal. Proceedings of the Australian Universities Quality Forum 2010, Gold Coast, Queensland, 80–88. Papadopoulos,T. (2010). Beyond discipline and technical knowledge: industry perspectives on the business curriculum. Industry and Higher Education, 24(2), 109–114. Popovic, C. & Thomas, C. (2009). Creating future proof graduates. Assessment Learning and Teaching Journal, 5 (Spring 2009), 37–39. Jackson, D., Sibson, R. & Riebe, L. (2013). Delivering work-ready business graduates-keeping our promises and evaluating our performance. Journal of Teaching and Learning for Graduate Employability, 4 (1), 2-22. 22 Price, M., O’Donovan, B., Rust, C. & Carroll, J. (2008). Assessment standards: A manifesto for change. Brookes E-Journal of Teaching and Learning, 2(3). Rae, D. (2007). Connecting enterprise and graduate employability: Challenges to the higher education culture and curriculum? Education + Training, 49(8/9), 605–619. Robley, W., Whittle, S. & Murdoch-Eaton, D. (2005). Mapping generic skills curricula: a recommended methodology. Journal of Further and Higher Education, 29(3), 221– 231. Rumsey, D. (2005). Implementing employability skills in training and assessment. Accessed by https://vetinfonet-staging.det.wa.edu.au/course/docs/employability_skills.pdf. Sadler, D. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment and Evaluation in Higher Education, 34(2), 159–179. Sciulli, N., Smith, M. & Ross, P. (2009). Benchmarking the first year accounting unit: Evidence from Australia. Asian Journal of Finance & Accounting, 1(2), E1. Smith, E. & Green, A. (2005). How workplace experiences while at school affect career pathways. Adelaide: National Centre for Vocational and Educational Research. Starkey, K. & Tempest, S. (2009). From crisis to purpose. Journal of Management Development, 28(8), 700–10. Taylor, A. (2005). What employers look for: The skills debate and the fit with youth perceptions. Journal of Education and Work, 18(2), 201–218. Tariq, V., Scott, E., Cochrane, A., Lee, M. & Ryles, L. (2004). Auditing and mapping key skills within university curricula. Quality Assurance in Education, 12(2), 70–81. Tertiary Education Quality and Standards Agency [TEQSA], (2011). Developing a framework for teaching and learning standards in Australian higher education and the role of TEQSA. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/teqsa/Documents/Teaching_Learni ng_Discussion_Paper.pdf Tight, M. (2010). Are academic workloads increasing? The post-war survey evidence in the UK. Higher Education Quarterly, 64(2), 200–215. Tomlinson, M. (2008). 'The degree is not enough': Students' perceptions of the role of higher education credentials for graduate work and employability. British Journal of Sociology of Education, 29(1), 49–61. Tripp, D. (1993). Critical incidents in teaching: Developing professional judgement. London, Routledge. Uchiyama, K. & Radin, J. (2008). Curriculum mapping in Higher Education: A vehicle for collaboration. Innovative Higher Education, 33(4), 271–280. Wheeler, J. (2008). The impact of social environments on emotional, social, and cognitive development. Journal of Management Development, 27(1), 129–145. Wittekind, A., Raeder, S. & Grote, G. (2010). A longitudinal study of determinants of perceived employability. Journal of Organizational Behavior, 1(4), 566–86. Yorke, M. & Knight, P. (2004). Embedding employability into the curriculum. Learning and Employability Series One, April 2006. UK: The Higher Education Academy. https://vetinfonet-staging.det.wa.edu.au/course/docs/employability_skills.pdf http://www.deewr.gov.au/HigherEducation/Policy/teqsa/Documents/Teaching_Learning_Discussion_Paper.pdf http://www.deewr.gov.au/HigherEducation/Policy/teqsa/Documents/Teaching_Learning_Discussion_Paper.pdf