June 2018, Vol. 10, No. 2 AJHPE 114 Research The contribution of discipline-independent cognitive skills to achieving success in higher education is increasingly acknowledged.[1-6] These skills are considered to be ‘important for individuals both as learners in foundation education and training, and as future employees in changing and flexible work roles’.[6] Referred to as key skills,[7] generic graduate attributes[8] or generic skills that underpin lifelong learning,[9] they usually relate to six broad categories of skills: number-based skills; communication skills; information and communication technology skills; the skills required to improve one’s own learning and performance; skills for problem-solving and skills for working with others.[4,10] Evidence is emerging that these underpinning generic learning skills may make an important contribution to academic performance in the first year at medical school.[11,12] Students experiencing academic difficulties in their first year at medical school report problems with information handling, problem solving, critical thinking and time management.[11] Academically at-risk medical students have been shown to have less practice, and confidence, in generic learning skills when compared with their peers on admission to university.[12] An academic support programme, purposefully designed to incorporate generic skills development, was found to close this ‘skills gap’ over a period of 12 months. These data suggest that generic skills proficiency may be a useful indicator of academic preparedness on entry to medical school. This may be particularly important in settings where widening participation in higher education is being pursued. A number of studies on the generic skills proficiency of medical school entrants have been based on self-reported data.[12-14] This may be considered such a significant limitation of the work as to render the findings uninformative to the broader academic community. Correlations between self-assessment of discipline-specific knowledge and/or skills and external measures of performance have been shown to be widely variable in many disciplines, including medicine, education, law, engineering, sports science, behavioural science, psychology, guidance counselling, dietetics, and the workplace.[15-20] The reasons why self-assessment of both domain-specific knowledge and discipline-specific skills is unreliable are well known; high performers tend to underestimate their ability, and poor performers lack both the required expertise and insight to recognise their lack of expertise, i.e. they don’t know what they don’t know.[17,20-22] What is unknown, however, is whether self-assessment of generic learning skills, which are not discipline-specific, is subject to the same major limitations. In South Africa (SA), 17 of 26 public higher education institutions currently use the National Benchmark Tests (NBTs), alongside the National Senior Certificate and other high school-leaving examination results, to admit students who are likely to succeed at university.[23-29] The NBTs are a set of criterion-referenced pre-university admission aptitude tests, similar to pre-admission aptitude tests written in the UK (UK Clinical Aptitude Test),[30-32] the USA (North American Medical College Admission Test),[33,34] Australia (Australian Graduate Medical School Admissions Test)[35] and Background. Strong generic learning skills may improve academic performance at medical school. Studies evaluating the generic learning skills proficiency of medical students use self-reported data. It is not known whether self-evaluation of discipline-independent skills exhibits the same problems of widely variable accuracy as self-assessment of discipline-related skills. Objective. To investigate whether the self-reported generic learning skills proficiency of medical school entrants was related to three objective measures of performance: pre-university admission aptitude-test scores, information technology (IT) proficiency on entry and early academic performance at university. Methods. This prospective study used a previously validated 31-item questionnaire to document the self-reported proficiency of medical school entrants (2011 - 2013) with regard to 6 categories of generic learning skills: information handling, technical and numeracy, computer, organisational, managing self-learning and presentation skills. The results of the questionnaire were compared with performance in pre-university admission aptitude tests, an IT placement test on entry and end-of-semester 1 examinations (after 6 months at university), which are the basis for promotion to semester 2. Results. A total of 640 of 648 (98.8%) students completed the questionnaire. Self-reported generic learning skills proficiency was found to be significantly related to pre-university admission aptitude test scores (medium effect size), IT proficiency on entry to university (large effect size) and early academic performance at university (small effect size). Academically weak students did not overestimate their skills proficiency. Conclusion. These findings support the opinion that self-reported generic skills proficiency can credibly contribute to determining the academic preparedness of medical school entrants. Afr J Health Professions Educ 2018;10(2):114-123. DOI:10.7196/AJHPE.2018.v10i2.971 Self-reported generic learning skills proficiency: Another measure of medical school preparedness V C Burch,1 MB BCh, MMed, PhD, FCP, FRCP; C N T Sikakana,2 BSc, PhD; G D Gunston,3 MB ChB, MPhil Ed; D Murdoch-Eaton,4 MBBS, MD, FRCPCH 1 Department of Medicine, Faculty of Health Sciences, University of Cape Town, South Africa 2 Division of Medical Biochemistry, Faculty of Health Sciences, University of Cape Town, South Africa 3 Department of Human Biology, Faculty of Health Sciences, University of Cape Town, South Africa 4 Medical School, Faculty of Medicine, Dentistry and Health, University of Sheffield, UK Corresponding author: G D Gunston (Geney.gunston@uct.ac.za) This open-access article is distributed under Creative Commons licence CC-BY-NC 4.0. 115 June 2018, Vol. 10, No. 2 AJHPE Research other countries such as Chile, Pakistan and Saudi Arabia.[36-38] These tests provide information about school-leavers that is supplementary to their pre-university academic achievements. As shown in Tables 1 A and B, the NBTs assess skills competency in three domains, namely academic literacy, quantitative literacy and mathematics.[29] The results are aggregated into three performance bands (proficient, intermediate and basic), which provide an indication of applicants’ academic preparedness and likely need for early academic support at university. The University of Cape Town (UCT) medical school accepts applicants who fall into the first two bands. Additionally, UCT uses a locally designed information technology (IT) placement test to identify medical school entrants who require additional intensive introductory IT training prior to starting semester 1. The semester 1 academic programme at UCT medical school consists of four courses: Chemistry, Physics, Introduction to Integrated Health Sciences (HUB1006F), and Becoming a Professional (BP).[39] HUB1006F introduces students to key physical, psychological, social and developmental concepts that shape the human life cycle from conception to death, via strategically designed problem-based learning cases. During this course, students gain an introductory overview of the human lifespan, and core discipline- specific knowledge and skills including anatomy, physiology, psychology and sociology. BP aims to promote the conduct, knowledge, attitudes and values associated with being a professional and a professional team member. Students develop a range of skills, including interpersonal, interviewing and leadership skills, in addition to critical analysis and reflection on professional conduct, diversity, health and human rights. End of semester-1 examination results comprise the results of these four courses. The purpose of this study was to determine whether the self-reported generic learning skills proficiency of medical school entrants was related to objective measures of performance, specifically: pre-university admission aptitude test scores (NBTs), IT proficiency on admission to university (IT placement test), and academic performance after the first 6 months at university (end of semester-1 examination results, which are the basis for promotion to semester 2). Demonstrating a relationship between self-reported generic learning skills proficiency and objective measures of performance would be of international interest because it would support the hypothesis that self- assessment of such skills, which are discipline-independent, may be a credible way of determining academic preparedness for university, and Table 1 B. Interpretation of benchmark levels in the three domain areas of the National Benchmark Tests (NBTs)[29] Benchmark performance band (level) Performance band (level) descriptor Proficient Performance in domain areas suggests that academic performance will not be adversely affected. If admitted, students should be placed on regular programmes of study. Intermediate Challenges in domain areas identified such that it is predicted that academic progress will be affected. If admitted, students’ educational needs should be met in a way deemed appropriate by the institution (e.g. extended or augmented programmes, special skills provision). Basic Serious learning challenges identified: it is predicted that students will not cope with degree-level study without extensive and long-term support, perhaps best provided through bridging programmes or FET colleges. Institutions registering students performing at this level would need to provide such support. FET = Further Education and Training. FET colleges offer vocational and occupational courses which provide education and training with a specific range of jobs or employment possibilities. Table 1 A. Skills assessed in the three domain areas of the National Benchmark Tests (NBTs)[29] Academic literacy Quantitative literacy Mathematics Making meaning from academic text Understanding vocabulary related to academic study Evaluating evidence used to support claims made by writers Extrapolating and drawing inferences and conclusions from text Differentiating main idea from supporting ideas in the overall and specific organisation of a passage Identifying text differences as related to the writers’ purposes, audiences and forms of communication Understanding how syntax and punctuation are used to express meaning Understanding basic numerical concepts used in text Applying quantitative procedures and reasoning in symbolic and non-symbolic situations Applying information from a variety of tables, graphs, charts and text Integrating information obtained from multiple sources Performing multiple-step calculations using information presented with text, symbols and graphs Identifying trends and patterns in various situations Applying properties of simple geometric shapes to determine measurements Interpreting quantitative information presented verbally, symbolically and graphically Understanding and applying properties of the real number system, including surds and exponents Recognising and using patterns, including sequences and series Applying relationships such as ratios and percentages in a variety of contexts Applying the results of algebraic manipulations with equations and inequalities Understanding the function concept and identifying properties of functions Interpreting transformations of functions represented algebraically or graphically Identifying relationships between graphs and their equations, or inequalities and the regions they describe Applying trigonometric identities and concepts in solving problems Understanding properties and interpreting representations of 2- and 3-dimensional shapes Applying principles of analytic geometry Interpreting various representations and measures of data Using logical skills in making deductions determining the validity of given assertions June 2018, Vol. 10, No. 2 AJHPE 116 Research the importance of generic learning skills in achieving success in higher education.[1,5,6] Methods Study participants This was a prospective study of students entering year 1 of the UCT MB ChB programme during 2011 - 2013. Survey instrument Data were collected using a 31-item generic learning skills questionnaire previously validated in the SA context and shown to be reliable (Cronbach’s α = 0.88).[14] The clustering of the skills into 6 categories (information- handling skills, technical and numeracy skills, computer skills, organisational skills, managing self-learning skills and presentation skills) was verified using factor analysis.[40] This questionnaire reports on learning skills proficiency in terms of: (i) frequency of practice of each of the 31 skills during the 12 months preceding entry into medical school, using a 4-point scale ranging from 1 (never) to 4 (every week), and (ii) level of self-confidence in performing these skills, using a 4-point rating scale from 1 (little or no experience), 2 (basic but I sometimes need help), 3 (enough for my needs) to 4 (more than I need, I often help others). Procedure On the first day of semester 1, at the end of the whole-class orientation session, all first-year medical students were fully briefed on the generic skills research project by the researcher, and given the opportunity to ask questions. Consenting students completed a specially designed paper-and- pencil version of the questionnaire, which was handed to the researcher prior to leaving the venue. The data from the completed self-assessment questionnaires were electronically captured using a digital scanning process, and imported into Excel (Microsoft, USA) spreadsheets for analysis. Student data included the NBT scores, IT placement test scores and end of semester-1 examination results. All student data were obtained from student records kept in the Undergraduate Office at UCT medical school, and entered onto an Excel spreadsheet for analysis. All spreadsheets were manually checked for completeness prior to commencing data analysis. Data analysis Comparisons were made between the generic skills proficiency of students defined in three categories, according to student performance: (i) likely academic performance at university, as defined by two NBT benchmark performance bands (Table 1 B): intermediate (likely to require additional academic support), v. proficient (unlikely to need additional academic support) (ii) IT proficiency, as determined by IT placement test performance: <60% = not proficient (requires intensive introductory training prior to start of formal classes), v. ≥60% = proficient (introductory training not needed) (iii) early academic performance at university, as reflected by end of semester-1 examination results: <60% (poor academic performance) v. ≥60% (good academic performance). Statistical analysis Descriptive statistical and correlation analyses were performed using GraphPad Prism 6 (GraphPad Software Inc., USA). Means were compared using the unpaired t-test; where data sets showed unequal variance, the Welch correction was used. A p-value <0.05 was considered significant. The effect size for the means compared was calculated using a pooled standard deviation, which took into account the difference in size of the groups compared.[41] The Human Research Ethics Committee of the Faculty of Health Sciences, UCT (ref. no. 509/2013), and the Educational Research Ethics Committee of the Faculty of Medicine and Health, University of Leeds, UK (ref. no. 0607/ DME/SKILLS), approved the study. Results A total of 640 of the 648 (98.8%) students enrolled during the study period completed the survey. Fig. 1 shows the interrelationships between self- reported generic skills proficiency and the three objective measures of performance (NBT, IT placement test and semester 1 examinations). The figure summarises the effect size data presented in Tables 2 - 5. Tables 2 A - C compare students in the intermediate and proficient NBT performance bands for academic literacy, quantitative literacy and mathematics, with respect to performance in the IT placement test and semester 1 examinations. Overall, students in the proficient band performed significantly better than those in the intermediate band. Eighty percent of the effect sizes (12/15) were large or very large (≥0.75), and 20% (3/15) were medium (0.45 - 0.74). Tables 3 A - C compare students in the intermediate and proficient NBT performance bands for academic literacy, quantitative literacy and mathematics with respect to frequency of practice of, and confidence in, six categories of generic learning skills. Mostly, students in the proficient band reported significantly more frequent practice. The exceptions were non-significant differences between students in the two bands with respect to the frequency of practice of managing self-learning skills for all NBT components, and information handling for mathematics. While most (72%) effect sizes were small, that for academic literacy in relation to computer National Benchmark Tests *Not relevant to the purpose of the study. Semester 1 examinations La rg e (T ab le s 2 A - C an d 4 A - D ) N ot evaluated* Very large (Tables 4 A - D) Sm al l ( Ta b le s 5 A - E) Generic skills pro�ciency Me diu m ( Tab les 3 A - C ) Medium (Tables 5 A - E) IT placement test Fig. 1. Interrelationships between generic skills proficiency and performance prior to admission to university (NBT performance), on admission to university (IT placement test) and after 6 months at university (end of semester-1 examinations). Overall effect size of significant relationships and reference to the relevant data tables in the text are included. 117 June 2018, Vol. 10, No. 2 AJHPE Research Table 2 A. Comparison of assessment outcomes for academic literacy: on entry (IT) and at end of semester 1 for MB ChB I students in intermediate and proficient NBT performance bands Assessment Mean score, % (95% CI) (n=640) Effect size, d* p-value†Intermediate (n=108) Proficient (n=532) Mean difference IT placement test 56.3 (51.6 - 60.9) 77.4 (76.2 - 78.6) 21.14 1.60 <0.001 Chemistry examination 54.2 (52.1 - 56.2) 62.3 (61.1 - 63.4) 8.08 0.63 <0.00 Physics examination 55.7 (53.6 - 57.9) 65.7 (64.5 - 66.9) 9.96 0.77 <0.001 HUB1006F examination 56.7 (55.1 - 58.4) 67.6 (66.8 - 68.3) 10.83 1.19 <0.001 BP examination 68.6 (67.5 - 69.8) 74.0 (73.5 - 74.5) 5.36 0.99 <0.001 NBT = National Benchmark Test; HUB1006F = Introduction to Health Sciences Part I; BP = Becoming a Professional. *Effect size: <0.2 very small, ≥ 0.2 small, ≥ 0.45 medium, ≥ 0.75 large and ≥ 76 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. Table 2 B. Comparison of assessment outcomes for quantitative literacy: on entry (IT) and at end of semester 1 for MB ChB I students in the intermediate and proficient NBT performance bands Assessment event Mean score, % (95% CI) (n=640) Intermediate (n=209) Proficient (n=431) Mean difference Effect size, d* p-value† IT placement test 64.3 (61.2 - 67.3) 78.8 (77.4 - 80.1) 14.50 1.06 <0.001 Chemistry examination 53.8 (52.4 - 55.3) 64.3 (63.1 - 65.6) 10.53 0.87 <0.001 Physics examination 55.6 (54.2 - 57.1) 68.1 (66.9 - 69.4) 12.52 1.03 <0.001 HUB1006F examination 59.8 (58.6 - 60.9) 68.6 (67.7 - 69.5) 8.82 0.97 <0.001 BP examination 70.6 (69.9 - 71.4) 74.3 (73.8 - 74.8) 3.68 0.67 <0.001 NBT = National Benchmark Test; HUB1006F = Introduction to Health Sciences Part I; BP = Becoming a Professional. *Effect size: < 0.2 very small, ≥ 0.2 small, ≥ 0.45 medium, ≥ 0.75 large and ≥ 1 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. Table 2 C. Comparison of assessment outcomes for mathematics: on entry (IT) and at end of semester 1 for MB ChB I students in intermediate and proficient NBT performance bands Assessment Mean score, % (95% CI) (n=640) Effect size, d* p-value†Intermediate (n=273) Proficient (n=367) Mean difference IT placement test 66.5 (64.1 - 68.9) 80.3 (79.0 - 81.7) 13.86 1.01 <0.001 Chemistry examination 53.3 (52.1 - 54.5) 66.5 (65.2 - 67.8) 13.24 1.17 <0.00 Physics examination 55.5 (54.3 - 56.7) 70.3 (69.1 - 71.7) 14.91 1.32 <0.001 HUB1006F examination 60.5 (59.5 - 61.5) 69.6 (68.7 - 70.6) 9.11 1.02 <0.001 BP examination 71.1 (70.5 - 71.7) 74.6 (74.0 - 75.2) 3.53 0.64 <0.001 NBT = National Benchmark Test; HUB1006F = Introduction to Health Sciences Part I; BP = Becoming a Professional. *Effect size: <0.2 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥76 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. Table 3 A. Comparison of mean generic learning skills category ratings for MB ChB I students in the intermediate and proficient NBT performance bands for academic literacy Mean rating of generic learning skills category (95% CI) (n=640) Generic learning skills category on entry Intermediate (n=108) Proficient (n=532) Mean difference Effect size, d* p-value Frequency of practice Information handling 3.18 (3.07 - 3.30) 3.46 (3.43 - 3.50) 0.28 0.60 <0.001† Technical and numeracy 2.92 (2.82 - 3.03) 3.16 (3.12 - 3.20) 0.24 0.48 <0.001† Computer skills 2.72 (2.56 - 2.87) 3.24 (3.20 - 3.29) 0.53 0.89 <0.001† Organisational skills 3.54 (3.44 - 3.63) 3.73 (3.69 - 3.77) 0.19 0.41 0.001 Managing self-learning 3.43 (3.35 - 3.51) 3.46 (3.43 - 3.50) 0.03 0.07 ns Presentation skills 2.67 (2.56 - 2.78) 2.90 (2.85 - 2.94) 0.22 0.43 <0.001 Confidence Information handling 2.74 (2.63 - 2.85) 3.14 (3.09 - 3.18) 0.40 0.76 <0.001 Technical and numeracy 2.68 (2.56 - 2.79) 3.02 (2.98 - 3.07) 0.35 0.65 <0.001 Computer skills 2.52 (2.37 - 2.67) 3.13 (3.07 - 3.18) 0.61 0.94 <0.001† Organisational skills 3.06 (2.95 - 3.17) 3.29 (3.24 - 3.34) 0.23 0.42 <0.001 Managing self-learning 3.09 (2.98 - 3.19) 3.21 (3.17 - 3.25) 0.12 0.26 <0.05† Presentation skills 2.46 (2.34 - 2.58) 2.87 (2.82 - 2.92) 0.41 0.73 <0.001 NBT = National Benchmark Test; ns = not significant. *Effect size: <0.2 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥1 very large. †Unequal variance: p-value recalculated using Welch correction for unequal variance. June 2018, Vol. 10, No. 2 AJHPE 118 Research Table 3 B. Comparison of mean generic learning skills category ratings for MB ChB I students in the intermediate and proficient NBT performance bands for quantitative literacy Mean rating of generic learning skills category (95% CI) (n=640) Generic learning skills category on entry Intermediate (n=108) Proficient (n=532) Mean difference Effect size, d* p-value† Frequency of practice Information handling 3.31 (3.24 - 3.38) 3.47 (3.43 - 3.51) 0.16 0.33 <0.001† Technical and numeracy 2.97 (2.90 - 3.05) 3.19 (3.15 - 3.24) 0.22 0.44 <0.001† Computer skills 2.88 (2.78 - 2.98) 3.28 (3.24 - 3.33) 0.41 0.68 <0.001† Organisational skills 3.63 (3.56 - 3.70) 3.73 (3.68 - 3.77) 0.09 0.20 <0.05 Managing self-learning 3.42 (3.36 - 3.48) 3.47 (3.43 - 3.51) 0.06 0.14 ns Presentation skills 2.75 (2.67 - 2.82) 2.91 (2.87 - 2.96) 0.17 0.32 <0.001 Confidence Information handling 2.85 (2.77 - 2.92) 3.18 (3.13 - 3.23) 0.33 0.63 <0.001 Technical and numeracy 2.74 (2.66 - 2.82) 3.07 (3.02 - 3.12) 0.33 0.63 <0.001† Computer skills 2.71 (2.60 - 2.81) 3.18 (3.12 - 3.23) 0.47 0.73 <0.001† Organisational skills 3.13 (3.05 - 3.21) 3.31 (3.26 - 3.36) 0.18 0.33 <0.001† Managing self-learning 3.11 (3.04 - 3.18) 3.23 (3.18 - 3.27) 0.11 0.25 <0.01 Presentation skills 2.60 (2.52 - 2.68) 2.90 (2.85 - 2.95) 0.30 0.52 <0.001 NBT = National Benchmark Test; ns = not significant. *Effect size: <0.2 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥1 very large. †Unequal variance: p-value recalculated using Welch correction for unequal variance. Table 3 C. Comparison of mean generic learning skills category ratings for MB ChB I students in the intermediate and proficient NBT performance bands for mathematics Mean rating of generic learning skills category (95% CI) (n=640) Generic learning skills category on entry Intermediate (n=108) Proficient (n=532) Mean difference Effect size, d* P-value Frequency of practice Information handling 3.38 (3.32 - 3.44) 3.44 (3.39 - 3.49) 0.06 0.12 ns† Technical and numeracy 3.03 (2.97 - 3.09) 3.19 (3.14 - 3.24) 0.16 0.32 <0.001 Computer skills 3.00 (2.90 - 3.07) 3.27 (3.22 - 3.33) 0.28 0.47 <0.001† Organisational skills 3.63 (3.57 - 3.69) 3.74 (3.70 - 3.79) 0.11 0.24 <0.05† Managing self-learning 3.44 (3.39 - 3.50) 3.46 (3.42 - 3.51) 0.02 0.05 ns Presentation skills 2.77 (2.71 - 2.84) 2.93 (2.87 - 2.98) 0.15 0.29 <0.001 Confidence Information handling 2.88 (2.82 - 2.90) 3.21 (3.16 - 3.26) 0.32 0.62 <0.001 Technical and numeracy 2.77 (2.70 - 2.84) 3.11 (3.06 - 3.16) 0.33 0.64 <0.001† Computer skills 2.78 (2.69 - 2.87) 3.21 (3.15 - 3.27) 0.43 0.66 <0.001† Organisational skills 3.14 (3.07 - 3.21) 3.33 (3.28 - 3.39) 0.19 0.35 <0.001 Managing self-learning 3.15 (3.08 - 3.21) 3.22 (3.18 - 3.26) 0.07 0.16 ns† Presentation skills 2.64 (2.57 - 2.71) 2.92 (2.86 - 2.98) 0.28 0.50 <0.001 NBT = National Benchmark Tests; ns = not significant. *Effect size: <0.2 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥1 very large. †Unequal variance: p-value recalculated using Welch correction for unequal variance. Table 4 A. Comparison of pre-university NBT assessment outcomes v. entry IT placement test and semester 1 assessment outcomes for Chemistry NBT domain IT placement test assessment on entry End of semester-1 assessment outcomes: Chemistry examination Mean percentage score (95% CI) (n=414) Effect size, d* p-value Mean percentage score (95% CI) (n=602) Effect size, d* p-value<60% (n=58 ) ≥60% (n=356) <60% (n=282) ≥60% (n=320) Academic literacy 61.6 (58.9 - 64.4) 74.8 (73.9 - 75.6) 1.58 <0.001† 70.0 (68.8 - 71.1) 75.9 (75.0 - 76.8) 0.65 <0.001† Quantitative literacy 54.7 (51.3 - 58.1) 74.0 (72.6 - 75.4) 1.45 <0.001 64.8 (63.2 - 66.4) 77.7 (76.4 - 79.0) 1.00 <0.001† Mathematics 51.6 (48.7 - 54.5) 65.2 (63.6 - 66.7) 0.94 <0.001† 57.0 (55.4 - 58.5) 72.3 (70.8 - 73.8) 1.16 <0.001 NBT = National Benchmark Tests. *Effect size: <0.1 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥ 1 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. 119 June 2018, Vol. 10, No. 2 AJHPE Research skills (6%) was large. The effect sizes for academic literacy in relation to information-handling and technical and numeracy skills, and those for quantitative literacy and mathematics in relation to computer skills (22%), were medium. In general, students in the proficient band reported significantly more confidence than those in the intermediate. The exception was the non-significant difference in confidence in managing self-learning skills between students in the two bands for mathematics. While most (55%) effect sizes were medium, those for academic literacy in relation to information-handling and computer skills (11%) were large. All effect sizes (33%) in relation to organisational skills and managing self-learning were small. Tables 4 A - D compare students’ results in the IT placement test or semester 1 examinations with their performance in the NBTs. Mostly, students who achieved a good pass (≥60%) in the IT placement test or semester 1 examinations had performed significantly better in the NBTs. Seventy-three percent of the effect sizes (11/15) were large or very large, and 27% (4/15) were medium. Tables 5 A - E compare students’ results in the IT placement test or semester 1 examinations with their frequency of practice of, and confidence in, the six generic learning skills categories. In general, students who achieved a good pass reported significantly more frequent practice. The exceptions were the non-significant differences between strong and weak performers in the frequency of practice of managing self-learning skills for the IT placement test or any semester 1 examinations, presentation skills for chemistry or HUB1006F and information-handling skills for chemistry, physics or HUB1006F. While most (87%) effect sizes were small, that for the IT placement test in relation to computer skills (3%) was very large. The effect sizes for the IT placement test in relation to technical and numeracy skills, and for HUB1006F and BP in relation to computer skills (10%) were medium. In general, students who achieved a good pass reported significantly more confidence. The exceptions were the non-significant differences in confidence in managing self-learning skills between strong and weak performers in the IT placement test or chemistry. Most effect sizes (63%) were small, but those for the IT placement test in relation to technical and numeracy skills and computer skills (7%) were large or very large. The effect sizes for the IT placement test in relation to information handling, organisational and presentation skills; physics in relation to information handling, technical and numeracy, and computer skills; HUB1006F in relation to computer skills; and BP in relation to information-handling and computer skills (30%) were medium. Correlation analyses were also performed for all the variables presented in Tables 2 - 5. In 87% (110/126) of comparisons, correlation coefficients Table 4 B. Comparison of pre-university NBT assessment outcomes v. entry IT placement test and semester 1 assessment outcomes for physics NBT domain IT placement test assessment on entry End of semester-1 assessment outcomes: Physics examination Mean percentage score (95% CI) (n=414) Effect size, d* p-value Mean percentage score (95% CI) (n=607) Effect size, d* p-value<60% (n=58 ) ≥60% (n=356) <60% (n=235) ≥60% (n=372) Academic literacy 61.6 (58.9 - 64.4) 74.8 (73.9 - 75.6) 1.58 <0.001† 69.7 (68.5 - 71.0) 75.3 (74.4 - 76.2) 0.61 <0.001 Quantitative literacy 54.7 (51.3 - 58.1) 74.0 (72.6 - 75.4) 1.45 <0.001 63.1 (61.5 - 64.8) 77.0 (75.6 - 78.3) 1.08 <0.001 Mathematics 51.6 (48.7 - 54.5) 65.2 (63.6 - 66.7) 0.94 <0.001† 55.3 (53.8 - 56.9) 71.0 (69.6 - 72.4) 1.19 <0.001† NBT = National Benchmark Tests. *Effect size: <0.1 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥1 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. Table 4 C. Comparison of pre-university NBT assessment outcomes v. entry IT placement test and semester 1 assessment outcomes for HUB1006F NBT domain IT placement test assessment on entry End of semester-1 assessment outcomes: HUB1006F examination Mean percentage score (95% CI) (n=414) Effect size, d* p-value Mean percentage score (95% CI) (n=635) Effect size, d* p-value<60% (n=58 ) ≥60% (n=356) <60% (n=170) ≥60% (n=465) Academic literacy 61.6 (58.9 - 64.4) 74.8 (73.9 - 75.6) 1.58 <0.001† 66.9 (65.3 - 68.5) 75.5 (74.8 - 76.2) 1.00 <0.001† Quantitative literacy 54.7 (51.3 - 58.1) 74.0 (72.6 - 75.4) 1.45 <0.001 63.5 (61.2 - 65.7) 75.0 (73.8 - 76.1) 0.86 <0.001 Mathematics 51.6 (48.7 - 54.5) 65.2 (63.6 - 66.7) 0.94 <0.001† 55.7 (53.8 - 57.7) 68.4 (67.1 - 69.8) 0.89 <0.001 NBT = National Benchmark Tests; HUB1006F = Introduction to Health Sciences Part I. *Effect size: <0.1 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥1 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. Table 4 D. Comparison of pre-university NBT assessment outcomes v. entry IT placement test and semester 1 assessment outcomes for BP NBT domain IT placement test assessment on entry End of semester-1 assessment outcomes: BP examination Mean percentage score (95% CI) (n=414) Effect size, d* P-value Mean percentage score (95% CI) (n=455) Effect size, d* P-value<60% (n=58 ) ≥60% (n=356) <60% (n=160) ≥60% (n=465) Academic literacy 61.6 (58.9 - 64.4) 74.8 (73.9 - 75.6) 1.58 <0.001† 67.8 (66.0 - 69.5) 75.3 (74.6 - 76.0) 0.85 <0.001† Quantitative literacy 54.7 (51.3 - 58.1) 74.0 (72.6 - 75.4) 1.45 <0.001 66.1 (63.7 - 68.5) 74.4 (73.5 - 75.6) 0.60 <0.001† Mathematics 51.6 (48.7 - 54.5) 65.2 (63.6 - 66.7) 0.94 <0.001† 59.7 (57.4 - 62.1) 67.4 (66.1 - 68.8) 0.52 <0.001 NBT = National Benchmark Tests; BP = Becoming a Professional. *Effect size: <0.1 is very small, ≥0.2 is small, ≥0.45 is medium, ≥0.75 is large and ≥1 is very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. June 2018, Vol. 10, No. 2 AJHPE 120 Research reflected the effect sizes as follows: small effect sizes had r-values <0.25; medium effect sizes had r-values 0.25 - 0.40, and large effect sizes had r-values >0.4 (data not shown). Discussion This study showed that the self-reported practice of and confidence in generic learning skills proficiency of first-year medical students was related to three objective measures of performance: pre-university admission aptitude test scores, IT proficiency on entry to university and early academic performance at university. Since these findings are based on self-assessment data, the credibility of which is often contested in the medical education literature,[15-20] it is essential to substantiate our findings before discussing their significance. Factors which are known to influence the accuracy of self-assessment data, including the nature of the self-assessment task, and the characteristics of the rating scales used,[42] are specifically addressed. Regarding the nature of the task, it is essential to recognise that the students in this study were asked to self-assess their generic learning skills proficiency rather than discipline-specific skills. This is important, because the challenges that students face when self-assessing discipline-specific skills were not relevant to this task. These challenges include students’ rudimentary understanding of the knowledge required to perform discipline-specific tasks proficiently,[22] the longstanding debate about the extent (breadth and depth) of profession-specific expertise required of medical graduates, i.e. curriculum content and overload,[43,44] and the observation that experts continue to display ‘disturbing discrepancies in their judgements of how much knowledge is enough’.[45] It is not, therefore, surprising that students may struggle to have a clear idea of discipline-specific proficiency and whether they have Table 5 A. Comparison of mean generic learning skills category ratings v. entry IT placement test results for MB ChB I students Generic learning skills category on entry Mean rating of generic learning skills category (95% CI) (n=414) Effect size, d* p-value<60% (n=58) ≥60% (n=356) Frequency of practice Information handling 3.30 (3.17 - 3.42) 3.45 (3.40 - 3.50) 0.34 <0.05 Technical and numeracy 2.95 (2.83 - 3.08) 3.19 (3.15 - 3.24) 0.53 <0.001 Computer skills 2.42 (2.22 - 2.62) 3.23 (3.17 - 3.28) 1.45 <0.001† Organisational skills 3.57 (3.46 - 3.68) 3.75 (3.70 - 3.79) 0.40 <0.01 Managing self-learning 3.48 (3.36 - 3.59) 3.45 (3.41 - 3.50) – 0.06 ns Presentation skills 2.73 (2.59 - 2.87) 2.89 (2.84 - 2.94) 0.34 <0.05 Confidence Information handling 2.75 (2.59 - 2.90) 3.13 (3.08 - 3.19) 0.72 <0.001 Technical and numeracy 2.65 (2.50 - 2.79) 3.04 (2.99 - 3.09) 0.78 <0.001 Computer skills 2.24 (2.06 - 2.42) 3.11 (3.05 - 3.17) 1.46 <0.001 Organisational skills 3.05 (2.91 - 3.18) 3.31 (3.26 - 3.37) 0.49 <0.001 Managing self-learning 3.09 (2.93 - 3.24) 3.20 (3.15 - 3.25) 0.24 ns Presentation skills 2.49 (2.34 - 2.63) 2.87 (2.82 - 2.93) 0.72 <0.001 ns = not significant. *Effect size: <0.2 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥1 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. Table 5 B. Comparison of mean generic learning skills category ratings v. end of semester-1 examination results for chemistry Generic learning skills category on entry Mean rating of generic learning skills category (95% CI) (n=602) Effect size, d* p-value<60% (n=282) ≥60% (n=320) Frequency of practice Information handling 3.36 (3.30 - 3.42) 3.43 (3.38 - 3.48) 0.14 ns† Technical and numeracy 3.03 (2.97 - 3.09) 3.15 (3.10 - 3.21) 0.25 <0.01 Computer skills 3.01 (2.93 - 3.09) 3.21 (3.15 - 3.27) 0.33 <0.001† Organisational skills 3.62 (3.57 - 3.68) 3.74 (3.69 - 3.79) 0.25 <0.01 Managing self-learning 3.43 (3.37 - 3.48) 3.48 (3.44 - 3.52) 0.13 ns Presentation skills 2.81 (2.75 - 2.87) 2.87 (2.81 - 2.92) 0.11 ns Confidence Information handling 2.96 (2.90 - 3.02) 3.16 (3.10 - 3.22) 0.36 <0.001 Technical and numeracy 2.83 (2.76 - 2.90) 3.05 (3.00 - 3.11) 0.42 <0.001 Computer skills 2.86 (2.77 - 2.95) 3.12 (3.05 - 3.19) 0.38 <0.001† Organisational skills 3.19 (3.12 - 3.25) 3.31 (3.25 - 3.37) 0.22 <0.01 Managing self-learning 3.15 (3.09 - 3.21) 3.21 (3.16 - 3.26) 0.13 ns Presentation skills 2.70 (2.64 - 2.77) 2.85 (2.79 - 2.92) 0.26 <0.01 ns = not significant. *Effect size: <0.2 is very small, ≥0.2 is small, ≥0.45 is medium, ≥0.75 is large and ≥1 is very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. 121 June 2018, Vol. 10, No. 2 AJHPE Research achieved it. The same, however, is not true of discipline-independent generic learning skills. Furthermore, the self-assessment process in this study reported on students’ prior experience of performing objective, well-defined, familiar activities, such as answering emails, or finding information on the internet. Self-assessment of such activities has been shown to be better aligned with objective performance data.[17,20,46,47] In addition, the skills self-assessed in this study were largely ‘observable’, and, like language proficiency and sports performance, can be more accurately self-assessed than cognitive skills such as clinical reasoning.[20] This point is well illustrated in our study, where sudents who performed poorly in the IT placement test appropriately rated themselves as less experienced and confident in their computer skills. This finding also suggests that poor performers may recognise their limitations if self-assessment tasks focus on familiar, observable, non-cognitive skills. A significant problem with self-assessment studies is the variable use of rating scales by participants.[42] In our study, this limitation was addressed by using rating scales that were comparative (to peers), quantitative and objectively anchored. Such scales have been shown to yield more robust self-assessment data.[46,47] Peer comparison probably served as an indirect source of feedback, which is known to further enhance the accuracy of self-assessment of skills.[17,46] For example, students who rated themselves as ‘know more than I need – I often assist others’ were more likely to have been approached repeatedly for help by peers who recognised their ability based on prior performance. Having addressed the key potential limitation of this study, the salient findings can now be discussed. Overall, they show that the relationship effect sizes for students’ self-reported confidence in their generic learning skills v. their academic results were greater than for self-reported frequency Table 5 C. Comparison of mean generic learning skills category ratings v. end of semester-1 examination results for physics Generic learning skills category on entry Mean rating of generic learning skills category (95% CI) (n=602) Effect size, d* p-value<60% (n=282) ≥60% (n=320) Frequency of practice Information handling 3.37 (3.30 - 3.44) 3.42 (3.38 - 3.47) 0.11 ns† Technical and numeracy 3.01 (2.95 - 3.08) 3.17 (3.12 - 3.22) 0.31 <0.001 Computer skills 2.99 (2.90 - 3.08) 3.20 (3.15 - 3.26) 0.35 <0.001† Organisational skills 3.61 (3.54 - 3.67) 3.74 (3.70 - 3.79) 0.29 <0.001† Managing self-learning 3.42 (3.36 - 3.47) 3.47 (3.43 - 3.51) 0.13 ns Presentation skills 2.77 (2.70 - 2.84) 2.89 (2.83 - 2.94) 0.21 <0.05 Confidence Information handling 2.91 (2.84 - 2.98) 3.17 (3.11 - 3.22) 0.48 <0.001 Technical and numeracy 2.78 (2.71 - 2.86) 3.07 (3.02 - 3.12) 0.55 <0.001† Computer skills 2.81 (2.72 - 2.91) 3.12 (3.06 - 3.18) 0.46 <0.001† Organisational skills 3.14 (3.07 - 3.21) 3.33 (3.27 - 3.38) 0.34 <0.001 Managing self-learning 3.13 (3.07 - 3.19) 3.22 (3.17 - 3.27) 0.19 <0.05 Presentation skills 2.67 (2.59 - 2.74) 2.87 (2.81 - 2.92) 0.35 <0.001 ns = not significant. *Effect size: <0.2 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥1 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. Table 5 D. Comparison of mean generic learning skills category ratings v. end of semester-1 examination results for HUB1006F Generic learning skills category on entry Mean rating of generic learning skills category (95% CI) (n=635) Effect size, d* p-value<60% (n=170) ≥60% (n=465) Frequency of practice Information handling 3.35 (3.27 - 3.44) 3.44 (3.40 - 3.48) 0.18 ns Technical and numeracy 3.03 (2.94 - 3.11) 3.16 (3.11 - 3.20) 0.25 <0.01† Computer skills 2.95 (2.83 - 3.06) 3.23 (3.18 - 3.27) 0.46 <0.001† Organisational skills 3.59 (3.51 - 3.66) 3.73 (3.69 - 3.77) 0.31 <0.001 Managing self-learning 3.43 (3.36 - 3.50) 3.47 (3.43 - 3.51) 0.09 ns Presentation skills 2.81 (2.73 - 2.89) 2.88 (2.83 - 2.93) 0.13 ns Confidence Information handling 2.94 (2.85 - 3.03) 3.12 (3.07 - 3.17) 0.34 <0.001 Technical and numeracy 2.83 (2.74 - 2.92) 3.01 (2.97 - 3.06) 0.34 <0.001 Computer skills 2.79 (2.67 - 2.91) 3.11 (3.05 - 3.17) 0.48 <0.001† Organisational skills 3.15 (3.07 - 3.23) 3.29 (3.24 - 3.34) 0.25 <0.01 Managing self-learning 3.12 (3.05 - 3.20) 3.21 (3.17 - 3.25) 0.19 <0.05 Presentation skills 2.68 (2.59 - 2.77) 2.85 (2.80 - 2.90) 0.30 <0.01 HUB1006F = Introduction to Health Sciences Part I; ns = not significant. *Effect size: <0.2 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥1 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. June 2018, Vol. 10, No. 2 AJHPE 122 Research of practice v. academic results. This makes sense, because practical skill proficiency is influenced by many factors other than frequency of practice. It is noteworthy that generally, the effect sizes were small for organisational skills, and largely insignificant for managing self-learning skills. This also makes sense, because such skills are unlikely to significantly influence aptitude test (NBTs) performance or IT proficiency, and the limited curriculum load in first year may not require well-developed organisational and self-learning management skills. These skills may, however, become more important in later years of study, where the large volume of work is likely to require them. This merits further exploration. Internationally, there is a call for more studies aimed at determining the content and format of academic support programmes that promote sustained academic success.[48] The work presented in this article supports a focus on generic learning skills development, in addition to discipline-specific knowledge and skills learning, in such programmes.[12] The questionnaire used in the study may facilitate the conceptualisation of academic support programmes that better suit students’ needs, and direct timeous allocation of extra resources on a needs rather than ad hoc basis that may jeopardise the sustainability of such programmes. Since the questionnaire is free and easy to administer, it may be particularly attractive in limited-resource settings where strategies for providing early academic support are likely to be most needed. While the findings of this study are limited to one institution, the results are encouraging, and the sample size was sufficiently large to provide meaningful data. This provides a clear mandate to conduct a multicentre study. The results also support the idea that self-assessed generic skills proficiency may be a welcome addition to university admissions and academic placement processes, to determine the academic preparedness of students from diverse backgrounds, and to further support efforts to improve the social mobility of all sectors of society.[49] Acknowledgements. The authors would like to acknowledge Vanessa Gray from the University of Leeds for her assistance with data processing. Author contributions. GDG gathered the generic learning skills data provided by consenting students on a specially designed paper-and-pencil version of the questionnaire. The data were electronically captured using a digital scanning process, and imported into Excel (Microsoft, USA) spreadsheets for analysis (DM-E). Student data (NBT scores, IT placement test scores and end of semester-1 examination results) were obtained from student records kept in the Undergraduate Office at UCT medical school, and entered onto an Excel spreadsheet for analysis. All spreadsheets were manually checked for completeness prior to commencing data analysis (VB and CS). All authors contributed to the review of literature, discussion of results and writing of the article. Funding. Travel expenses for DM-E and some administrative and data processing in the UK were supported by DM-E’s National Teaching Fellowship awarded by the Higher Education Academy, UK. Conflicts of interest. None. 1. Dearing R. Higher Education in the Learning Society: Report of the National Committee of Inquiry into Higher Education. London: Department for Education and Employment, 1997. 2. Credé M, Kuncel NR. Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspect Psychol Sci 2008;3(6):425-453. https://doi.org/10.1111/j.1745-6924.2008.00089.x 3. Collin VT, Violato C, Hecker K. Aptitude, achievement and competence in medicine: A latent variable path model. Adv Health Sci Educ 2009;14(3):355-366. https://doi.org/10.1007/s10459-008-9121-7 4. Jones A. Generic attributes as espoused theory: The importance of context. High Educ 2009;58(2):175-191. https://doi.org/10.1007/s10734-008-9189-2 5. Frenk J, Chen L, Bhutta ZA, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet 2010;376(9756):1923-1958. https://doi.org/10.1016/S0140- 6736(10)61854-5 6. Green A. Core skills, key skills and general culture: In search of the common foundation in vocational education. Eval Res Edu 1998;12(1):23-43. https://doi.org/10.1080/09500799808666929 7. Washer P. Revisiting key skills: A practical framework for higher education. Qual High Educ 2007;13(1):57-67. http://doi.org/10.1080/13538320701272755 8. Barrie S. A conceptual framework for the teaching and learning of generic graduate attributes. Stud High Educ 2007;32(4):439-458. https://doi.org/10.1080/03075070701476100 9. Murdoch-Eaton D, Whittle S. Generic skills in medical education: Developing the tools for successful lifelong learning. Med Educ 2012;46(1):120-128. https://doi.org/10.1111/j.1365-2923.2011.04065.x 10. Mayer E, Australian Educational Council, Mayer Committee. Putting general education to work: The key competencies report. Melbourne: Australian Education Council and Ministers of Vocational Education, Employment and Training, 1992. http://hdl.voced.edu.au/10707/72980 (accessed 11 June 2018). 11. Paul G, Hinman G, Dottl S, Passon J. Academic development: A survey of academic difficulties experienced by medical students and support services provided. Teach Learn Med 2009;21(3):254-260. https://doi. org/10.1080/10401330903021041 12. Burch VC, Sikakana CNT, Gunston G, Shamley DR, Murdoch-Eaton D. Generic learning skills in academically- at-risk medical students: A development programme bridges the gap. Med Teach 2013;35(8):671-677. https://doi. org/10.3109/0142159X.2013.801551 13. McLean M, Shaban S, Murdoch-Eaton D. Transferable skills of incoming medical students and their development over the first academic year: The United Arab Emirates experience. Med Teach 2011;33(6):e297-e305. https://doi. org/10.3109/0142159X.2011.565826 14. Murdoch-Eaton D, Manning D, Kwizera E, Burch V, Pell G, Whittle S. Profiling undergraduates’ generic learning skills on entry to medical school: An international study. Med Teach 2012;34(12):1033-1046. https://doi.org/10 .3109/0142159X.2012.706338 Table 5 E. Comparison of mean generic learning skills category ratings v. end of semester-1 examination results for BP Generic learning skills category on entry Mean rating of generic learning skills category (95% CI) (n=615) Effect size, d* p-value<60% (n=160) ≥60% (n=465) Frequency of practice Information handling 3.25 (3.17 - 3.34) 3.46 (3.42 - 3.50) 0.44 <0.001† Technical and numeracy 3.06 (2.97 - 3.14) 3.17 (3.12 - 3.21) 0.22 <0.05† Computer skills 2.91 (2.79 - 3.03) 3.22 (3.17 - 3.27) 0.51 <0.001† Organisational skills 3.54 (3.45 - 3.62) 3.74 (3.70 - 3.78) 0.44 <0.001† Managing self-learning 3.40 (3.34 - 3.46) 3.46 (3.43 - 3.50) 0.15 ns Presentation skills 2.77 (2.68 - 2.86) 2.89 (2.84 - 2.94) 0.23 <0.05 Confidence Information handling 2.90 (2.80 - 2.99) 3.14 (3.09 - 3.19) 0.45 <0.001† Technical and numeracy 2.89 (2.81 - 2.98) 3.01 (2.96 - 3.06) 0.21 <0.05 Computer skills 2.78 (2.66 - 2.91) 3.10 (3.04 - 3.16) 0.48 <0.001† Organisational skills 3.09 (3.01 - 3.17) 3.30 (3.25 - 3.35) 0.38 <0.001 Managing self-learning 3.10 (3.03 - 3.18) 3.21 (3.17 - 3.26) 0.24 <0.01 Presentation skills 2.67 (2.57 - 2.76 2.84 (2.79 - 2.90) 0.30 <0.01 BP = Becoming a Professional; ns = not significant. *Effect size: <0.2 very small, ≥0.2 small, ≥0.45 medium, ≥0.75 large and ≥1 very large. †Unequal variance; p-value recalculated using Welch correction for unequal variance. https://doi.org/10.1007/s10459-008-9121-7 https://doi.org/10.1007/s10734-008-9189-2 https://doi.org/10.1016/S0140-6736(10)61854-5 https://doi.org/10.1016/S0140-6736(10)61854-5 https://doi.org/10.1111/j.1365-2923.2011.04065.x https://doi.org/10.1080/10401330903021041 https://doi.org/10.1080/10401330903021041 https://doi.org/10.3109/0142159X.2013.801551 https://doi.org/10.3109/0142159X.2013.801551 https://doi.org/10.3109/0142159X.2011.565826 https://doi.org/10.3109/0142159X.2011.565826 https://doi.org/10.3109/0142159X.2012.706338 https://doi.org/10.3109/0142159X.2012.706338 123 June 2018, Vol. 10, No. 2 AJHPE Research 15. Falchikov N, Boud D. Student self-assessment in higher education: A meta-analysis. Rev Educ Res 1989;59(4):395- 430. https://doi.org/10.3102/00346543059004395 16. Gordon MJ. A review of validity and accuracy of self-assessments in health professions training. Acad Med 1991;66(12):762-769. 17. Dunning D, Heath C, Suls JM. Flawed self-assessment: Implications for health, education and the workplace. Psych Science Public Interest 2004;5(3):69-106. https://doi.org/10.12691/ajrd-5-3-3 18. Eva KW, Regehr G. Self-assessment in the health professions: A reformulation and research agenda. Acad Med 2005;80(Suppl 10):S46-S54. 19. Davis DA, Mazmanian PE, Fordis M, Harrison RV, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence. A systematic review. JAMA 2006;296(9):1094-1102. https:// doi.org/10.1001/jama.296.9.1094 20. Zell E, Kristan Z. Do people have insight into their abilities? A metasynthesis. Perspect Psychol Sci 2014;9(2):111- 125. https://doi.org/10.1177/1745691614521244 21. Dunning D, Johnson K, Ehrlinger J, Kruger J. Why people fail to recognize their own incompetence. Curr Dir Psychol Sci 2003;12(3):83-87. https://doi.org/10.1111/1467-8721.01235 22. Eva KW, Cunnington JPW, Reiter HI, Keane DR, Norman GR. How can I know what I don’t know? Poor self-assessment in a well-defined domain. Adv Health Sci Educ 2004;9(3):211-214. https://doi.org/10.1023/ B:AHSE.0000038209.65714.d4 23. South Africa. Department of Education White Paper 3: A programme for the transformation of higher education. http://www.che.ac.za/sites/default/files/publications/White_Paper3.pdf (accessed 11 June 2018). 24. Griesel H. Access and Entry Level Benchmarks: The National Benchmark Tests Project. Pretoria: Higher Education South Africa (HESA), 2006. 25. Rankin N, Schöer V, Sebastiao C, Van Walbeek C. Predictors of academic performance: National Senior Certificate v. National Benchmark Tests. S Afr J High Educ 2012;26(3):564-585. 26. Wilson-Strydom M. Using the NBTs to inform institutional understandings of ‘under-preparedness’: Implications for admissions criteria. S Afr J High Educ 2012;26(1):136-151. 27. Cliff A. The National Benchmark Test in academic literacy: How might it be used to support teaching in higher education? Lang Matters 2015;46(1):3-21. https://doi.org/10.1080/10228195.2015.1027505 28. Wadee AA, Cliff A. Pre-admission tests of learning potential as predictors of academic success of first-year medical students. S Afr J High Educ 2016;30(2):264-278. http://doi.org/10.20853/30-2-619 29. Centre for Educational Testing for Access and Placement. What is in the NBTs? 2016. http://www.nbt.ac.za/ content/what-nbts (accessed 11 June 2018). 30. McManus IC, Smithers E, Partridge P, Keeling A, Fleming PR. A Levels and intelligence as predicators of medical careers in UK doctors: 20 year prospective study. BMJ 2003;327(7407):139-142. https://doi. org/10.1136%2Fbmj.327.7407.139 31. Adam J, Dowell J, Greatrix R. Use of UKCAT scores in student selection by UK medical schools, 2006 - 2010. BMC Med Educ 2011;11:98. https://doi.org/10.1186/1472-6920-11-98 32. McManus IC, Dewberry C, Nicholson S, Dowell JS. The UKCAT-12 study: Educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross- sectional collaborative study of 12 UK medical schools. BMC Med 2013;11:244. https://doi.org/10.1186/1741- 7015-11-244 33. Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research. Acad Med 2007;82(1):100-106. https://doi.org/10.1097/01.ACM.0000249878.25186.b7 34. Callahan CA, Hojat M, Velsoki J, Erdmann JB, Gonnella JS. The predictive validity of the three versions of the MCAT in relation to performance in medical school, residency, and licencing examinations: A longitudinal study of 36 classes of Jefferson Medical College. Acad Med 2010;85(6):980-987. https://doi.org/10.1097/ ACM.0b013e3181cece3d 35. Coates H. Establishing the criterion validity of the Graduate Medical School Admissions Test (GAMSAT). Med Educ 2008;42(10):999-1006. https://doi.org/10.1111/j.1365-2923.2008.03154.x 36. Contreras R, Ubilla S, Ugalde H, Vincentini E. Comparative study of the academic aptitude test and final grading in Chilean medical schools. Rev Med Chile 1984;112(10):1033-1043. 37. Al Alwan I, Al Kushi M, Magzoub M, Elzubeir M. Health sciences and medical college preadmission criteria and prediction of in-course academic performance: A longitudinal study. Adv Health Sci Educ 2013;18(3):427-438. https://doi.org/10.1007/s10459-012-9380-1 38. Khan JS, Tabasum S, Mukhtar O. Comparison of pre-medical academic achievement, entrance test and aptitude test scores in admission selection process. J Pakistan Med Assoc 2013;63(5):552-557. 39. University of Cape Town. Faculty of Health Sciences Undergraduate Studies: 2016. Handbook 8a in this series of handbooks. Rules And Curricula For Undergraduate Programmes Bachelor of Medicine and Bachelor Of Surgery (MBChB) pp 24-25. http://www.uct.ac.za/usr/downloads/uct.ac.za/apply/handbooks/Handbook8A_ HealthSciencesUndergraduate_2016.pdf (accessed 11 June 2018). 40. Whittle SR, Pell G, Murdoch-Eaton DG. Recent changes to students’ perceptions of their key skills on entry to higher education. J Further High Ed 2010;34(12):557-570. https://doi.org/10.3109/0142159X.2012.706338 41. Lenhard W, Lenhard A. Calculation of Effect Sizes. Bibergau: Psychometrica, 2016. http://www.psychometrica. de/effect_size.html. (accessed 11 June 2018). 42. Ward M, Gruppen L, Regehr G. Measuring self-assessment: Current state of the art. Adv Health Sci Educ 2002;7(1):63-80. 43. Anderson J, Graham A. A problem in medical education: Is there an information overload? Med Educ 1980;14(1):4-7. 44. Jones R, Higgs R, de Angelis C, Prideaux D. Changing face of medical curricula. Lancet 2001;357(9257):699-703. https://doi.org/10.1016/S0140-6736(00)04134-9 45. Bergman EM, Prince KJ, Drukker J, van der Vleuten CP, Scherpbier AJ. How much anatomy is enough? Anat Sci Educ 2008;1(4):184-188. https://doi.org/10.1002/ase.35 46. Goffin RD, Olson JM. Is it all relative? Comparative judgements and the possible improvement of self-ratings and ratings of others. Perspect Psychol Sci 2011;6(1):48-60. https://doi.org/10.1177/1745691610393521 47. Freund PA. How smart do you think you are? A meta-analysis on the validity of self-estimates of cognitive ability. Psychol Bull 2012;138(2):296-321. https://doi.org/10.1037/a0026556 48. Cleland J, Leggett H, Sandars J, Costa MJ, Patel R, Moffat M. The remediation challenge: Theoretical and methodological insights from a systematic review. Med Educ 2013;47(3):242-251. https://doi.org/10.1111/ medu.12052 49. Milburn A. Fair Access to Professional Careers: A Progress Report by the Independent Reviewer on Social Mobility and Child Poverty. London: The National Archives, 2012. https://www.gov.uk/government/uploads/ system/uploads/attachment_data/file/61090/IR_FairAccess_acc2.pdf (accessed 11 June 2018). Accepted 5 October 2017. https://doi.org/10.12691/ajrd-5-3-3 https://doi.org/10.1001/jama.296.9.1094 https://doi.org/10.1001/jama.296.9.1094 https://doi.org/10.1111/1467-8721.01235 https://doi.org/10.1023/B:AHSE.0000038209.65714.d4 https://doi.org/10.1023/B:AHSE.0000038209.65714.d4 http://www.che.ac.za/sites/default/files/publications/White_Paper3.pdf http://doi.org/10.20853/30-2-619 http://www.nbt.ac.za/content/what-nbts http://www.nbt.ac.za/content/what-nbts https://doi.org/10.1136%2Fbmj.327.7407.139 https://doi.org/10.1136%2Fbmj.327.7407.139 https://doi.org/10.1186/1472-6920-11-98 https://doi.org/10.1186/1741-7015-11-244 https://doi.org/10.1186/1741-7015-11-244 https://doi.org/10.1097/ACM.0b013e3181cece3d https://doi.org/10.1097/ACM.0b013e3181cece3d https://doi.org/10.1111/j.1365-2923.2008.03154.x https://doi.org/10.1007/s10459-012-9380-1 http://www.uct.ac.za/usr/downloads/uct.ac.za/apply/handbooks/Handbook8A_HealthSciencesUndergraduate_2016.pdf http://www.uct.ac.za/usr/downloads/uct.ac.za/apply/handbooks/Handbook8A_HealthSciencesUndergraduate_2016.pdf http://www.psychometrica.de/effect_size.html. http://www.psychometrica.de/effect_size.html. https://doi.org/10.1016/S0140-6736(00)04134-9 https://doi.org/10.1111/medu.12052 https://doi.org/10.1111/medu.12052 https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/61090/IR_FairAccess_acc2.pdf https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/61090/IR_FairAccess_acc2.pdf