Radiology resident selection Radiology resident selection Objective Restructured Interview to assess five essential attributes *Neela Lamki, Alfred B. Watson, Richard G. Fisher SQU JOURNAL FOR SCIENTIFIC RESEARCH: MEDICAL SCIENCES 2003 VOL 5, NO. –2, 27–30 ©SULTAN QABOOS UNIVERSIT Y R E V I E W Dept of Radiology, Baylor College Of Medicine, 1 Baylor Plaza, Houston, Texas 77030, USA. *To whom correspondence should be addressed. Email: nlamki@bcm.tmc.edu ABSTRACT. Like in other medical fields, candidates seeking a career in Radiology requires special skills and aptitudes. Selecting can- didates for radiology residency is made difficult by the fact that many of the essential qualities predictive of a good radiology consultant, such as interpersonal skills, recognition of limits, curiosity, conscientiousness, and confidence level, are “non-cognitive”, and thus difficult to assess. This paper describes the selection procedure developed by the Department of Radiology of Baylor College of Medicine to measure, as objectively as possible, both the cognitive and non-cognitive qualities of candidates, based on a combination of traditional screening and Objective Structured Interviews. This paper highlights efficacy of this selection procedure that includes both cognitive and non-cogni- tive factors, that is relevant also to other medical specialities. Key words: radiology, residency, cognitive, non-cognitive, Objective Structured Interview SELECTION OF RESIDENTS FOR TRAINING IN ANY specialty of medicine is very difficult. The difficul-ties are compounded when it comes to selection of residents for specific specialties. Radiology is one of the specialties that have special needs, with respect to the essential attributes that the future resident needs to have. In addition, the particular location, namely the university and the hospital where the resident will be working, also contributes to the specificity of the needs. It is generally accepted that criteria for selection of residents include, on one hand, indicators of their academic performance and on the other, such attributes that relate to attitude and behaviour. In the present paper, these two domains are termed cognitive and non-cognitive factors. In this sense, “cognitive” factors include a candidate’s medical school grades, national board examination scores, rank order, etc, while the “non-cognitive” factors refer to the personal characteristics such as conscientiousness and interpersonal skills. The cognitive factors are generally easier to assess than the non-cognitive ones. Although both factors determine the quality and performance of the resident and of the future radiologist, it is generally accepted that the non-cognitive factors are better indi- cators than the cognitive factors.1–7 For example, medi- cal school performance has been shown to be of little predictive value in the performance and quality of a resi- dent.8,9,10 Nevertheless, both are needed in the selection of ideal residents. This paper seeks to demonstrate a more objective method we have developed at Baylor College of Medicine in Houston for the selection of radiology residents. It is based on structured interviews designed to enable the interviewers to select the best seven candidates every year out of the 70–90 interviewees. These interviewees are selected from about 350 candidates who apply annu- ally for the 7 available positions. Our radiology training programme has a total of 28 residents in a four-year pro- gramme, not counting the obligatory clinical year. Neela Lamki Radiology Selection ������������������������� ������������������������������������������ �������������������������������� ��������:����������������������������������������������������������������������������������� ������������������������������������������������������������������������������������� :���������� �����������������������������������������������������������������������.������������������������������� ������������������������������������������������������������������������������������������������������� ���������������������������������������������. 28 L A M K I E T A L Because most candidates who apply to Baylor College of Medicine are la crème de la crème, it is particularly difficult to select from them the magnificent seven most suited for the positions. We select residents based on both cognitive and non-cognitive attributes. The cog- nitive attributes are generally extractible from the can- didate’s application, while the non-cognitive attributes are elicited mainly at the interview. The question then arises as to what constitutes a good interview and how the interview can be made more objective.10–13 To elicit the non-cognitive attributes of the applicants, we have developed a specially structured interview. The inter- view was structured to assess the five attributes that we decided were essential for candidates of the Residency Programme at Baylor and its hospitals. The Selection Committee of the Radiology residency programme reviewed the literature, specifically the med- ical educational literature, and came to the conclusion that there is indeed a very good correlation between the candidates’ attributes and behaviours in post-graduate performance.5,14–16 In this paper we describe our expe- rience with this structured interview and the attributes selected. M E T H O D Initial screening of the applicants is performed by two members of the Resident Selection Committee by screening the applications. Typically, these members are the Director and the Assistant Director of Radiology Education. The important factors considered in the screening process are the common cognitive ones such as () United States Medical Licensing Examination (USMLE ) scores, (2) medical school grades, including (a) basic science, and (b) more importantly, performance in clinical rotation, (3) the letter from the dean, (4) three letters of references, and (5) any other obvious factors. To evaluate the more non-cognitive factors, we chose the five attributes that were deemed most appropriate for training radiologists at our institution. Baylor College of Medicine uses a county hospital as a main base for the residents, with rotations in five other semi-private hospitals for the radiology training. Hence the residency makes specific demands on the residents. The non-cog- nitive attributes that were thought to be essential to a radiology resident’s performance at Baylor College of Medicine Radiology Residency Training Programme were accordingly selected. These were: () interpersonal skills, (2) recognition of limits, (3) curiosity, (4) consci- entiousness, and (5) confidence level. Some of these attributes were derived from an exten- sive review of the literature and a combination of studies. One such study carried out structured interviews with experienced radiology faculty to find out what would constitute a good radiologist.16 Some of these attributes have been found to be stable for over a decade by inter- views with radiologists.14 The findings were similar in a study that involved three different institutions.15 Our interviewers are given specific tasks, namely, to ask questions and present scenarios that can test the spe- cific attributes of the applicants. Sample questions are distributed to the interviewers as a guide. The Resident Selection Committee of the Department of Radiology has developed these sample questions designed to explore each of the five attributes. The five essential attributes are distributed among three faculty interviewers. Two of the interviewers are given the responsibility of exploring two attributes each, and the third interviewer is given only one attribute to explore. Each faculty interviewer is instructed to note response to the questions related to the attribute he/she is responsible for. He/she is asked to rank the candidates individually from  to 5; a score of  being unacceptable, and 5 being well-above aver- age. Interviews are set for 30 minutes each, and all inter- views are slated for the same afternoon. Each candidate is thus interviewed by three faculty and one resident in the same afternoon. Typically 6–8 candidates are interviewed in one day during the interview season. The committee members, who are not formally interviewing candidates on that day, assess candidates during the lunch gathering. The chief resident assesses them during the hospital tours that he/she conducts. DEFINITIONS OF THE ATTRIBUTES SELEC TED . Interpersonal skills • Ability to interact effectively (orally and in writing) with patients • Ability to interact effectively with faculty and clinicians • Ability to react appropriately with the interviewer 2. Recognition of limits • Ability to identify a problem situation and its poten- tial problems • Recognition of lack of knowledge or skill • Willingness to seek assistance when needed 3. Curiosity • The ability to independently seek knowledge • An intrinsic interest in medicine or science beyond direct patient care • An interest in radiology and our programme 29 4. Conscientiousness • Thoroughness in work • Willingness to go above and beyond the call of duty • Ability to assertively put forth one’s own opinion 5. Confidence • Make decisions and implement choices appropriate to training level and education • Ability to appropriately assess situations and priori- tise tasks • Aggressively seek opportunities for trying new skills SAMPLE QUESTIONS ASKED AT THE INTERVIEW . Interpersonal skills • Candidate’s relationship with friends; what they do together at leisure and other times, as well as their strengths and weaknesses • Jobs held that interact with others and choice of co- workers • Attitudes towards being yelled at mistakenly by an attending surgeon 2. Recognition of limits • The candidate is asked to describe a situation in which the candidate felt like he/she was over his/her head. What got him/her into this situation and what did he/she did to get out of it? • The candidate is asked to express his/her limitations and weaknesses and what his/her efforts to work through them. • In a scenario that a patient is dying and the candidate is alone, would he/she do something for the patient that may exceed his/her limitations? 3. Curiosity • How does a candidate arrive at a solution in a situa- tion of a very puzzling patient’s ailment? • The elective courses the candidate chose at the University and whether they were enjoyable and why? To describe his/her research project if any. • Why is the candidate interested in Baylor radiology programme? 4. Conscientiousness • An example of a deed that the candidate had done that might be considered “above and beyond” the call of duty • Action taken after committing an error • How many times has he/she researched a disease proc- ess when not presenting the case at a conference? R A D I O L O G Y R E S I D E N T S S E L E C T I O N 5. Confidence • Ability of the candidate to make decisions under dif- ficult situations. • Assessment of the candidate’s self worth, and self- confidence. • Ability of the candidate to facilitate appropriate diagnostic examination choices by the referring clinicians. D A T A A N A L Y S I S A N D S E L E C T I O N P R O C E S S All interviewers, faculty and resident interviewers meet at the end of each interview day for about 90 minutes to discuss the 6–8 candidates interviewed that day. Each interviewer discusses the ranking of the attributes he/ she is responsible for assessing. All the five attributes are evaluated and weighted accordingly. Additional input from informal assessment by the chief resident, the non- interviewing faculty, and the resident interviewers are obtained and weighted accordingly. The cognitive and other standard hard data, e.g., grades, board scores, deans’ letters and letters of recommendations are revis- ited and reanalysed at this meeting. This group of indi- viduals who have assessed the applicant then comes to a consensus as to the relative importance of each fac- tor among the cognitive and the non-cognitive factors as they apply to each individual candidate. An over- all score is assigned to the candidate on a scale of  to 0 by the Radiology Selection Committee. At the end of the season, the scores of all candidates are reviewed and the best of the group are selected for the “National Residency Match Programme.” D I S C U S S I O N Preliminary results indicate that the use of the essential attributes-oriented interviews lead to a more standard- ised structure during the interview. Interviewers who were initially uneasy about the standardised format, have now eventually accepted the idea. Indeed, they find that it is easier to elicit important personality characteristics and with greater intensity and objectivity. Experienced faculty interviewers find that these attribute-oriented interviews are more focused than the standard inter- views, and they facilitate better comparison of can- didates particularly with respect to the all-important non-cognitive factors. The interviews are found to be more equitable. However, the exact weighting of the fac- tors to be considered has yet to be defined, and this will require a larger data base. After only the first two years of implementation, both the interviewers and the Resident Selection Committee 30 found this method to be more objective. It has now been five years since we started implementing this, and so far, it appears that we did manage to select better residents, at least from personality point of view. Cursory com- parison with historic data to date indicates that there are less personality clashes and interpersonal problems arising from the residents—among themselves or with the staff. However, there is not enough data to analyse this more critically at this stage. Likewise, for the other attributes, the impression of the teaching faculty at the Baylor Residency Programme is that we now have a bet- ter group of residents—specifically with respect to the traits related to the 5 attributes. The residents tend to be more conscious of their limits, without losing confidence. They are more conscientious in their work and particu- larly when on night-duty call. They tend to research the literature in whatever they learn at conferences or from one-to-one teaching. They are more involved in research projects as evidenced by presentations at scientific meet- ings and publications in these last five years compared to before the implementation of the attribute-oriented interview. A larger pool of residents and a longer follow- up period will be required for a more accurate and statis- tically valid analysis of the relative contributions of each of the 5 attributes. There are several factors that play a role in the ideal performance of the resident during the residency, and also later as a consultant radiologist. The future perform- ance of these residents as radiology consultants is the ultimate test of any resident selection criteria. Criteria need to be established to assess not only the resident’s performance during the 4 years of radiology residency period but also the performance of the trainee later on as a consultant radiologist, and the position achieved. A further study will ascertain the success of this new Attribute-oriented selection process that we are propos- ing. More time is needed to appropriately evaluate this concept in resident selection. R E F E R E N C E S . Adusumilli S, Cohan RH, Marshall KW, Fitzgerald JT, Oh MS, Gross BH, et al. How well does applicant rank order predict subsequent performance during radiology residency? Acad Radiol, 2000, 7, 635–40. 2. Baker JD, Bailey MK, Brahen NH, Conroy J M, Dorman B H, Haynes G R. Selection of Anesthesiology Residents. Acad Med, 993, 68, 6–63. 3. Smith WL, Berbaum KS. Improving Resident Selection: Dis- crimination by Perceptual Abilities. Invest Radiol, 99, 26, 90–92. 4. Johnson EK, Edwards JC. Current practices in admission interviews at U.S. medical schools. Acad Med, 99, 66, 408– 42. 5. Wood PS, Smith WL, Altmaier EM, Tarico VS, Franken EA Jr. A prospective study of cognitive and noncognitive selection criteria as predictors of resident performance. Invest Radiol, 990, 25, 855–859. 6. Garden FH, Smith BS. Criteria for selection of physical medi- cine and rehabilitation residents. Am J Phys Med Rehabil, 989, 68, 23–27. 7. Komives E, Weiss ST, Rosa RM. The applicant interview as a predictor of resident performance. J Med Educ, 984, 59, 425– 426. 8. Wagoner NE, Suriano JR, Stoner JA. Factors used by program directors to select residents. J Med Educ, 986, 6, 0–2. 9. Sade RM, Stroud MR, Levine JH, Fleming GA. Criteria for selection of future physicians. Ann Surg, 985, 20, 225–230. 0. Boyse TD, Patterson SK, Cohan RH, Korobkin M, Fitzger- ald JT, Oh MS, et al. Does medical school performance pre- dict radiology resident performance? Acad Radiol, 2002, 9, 437–445. . McDonald JS, Lingam RP, Gupta B, Jacoby J, Gough HG, Bradley P. Psychologic testing as an aid to selection of resi- dents in anesthesiology. Anesth Analg, 994, 78, 542–547. 2. Gunderman RB, Jackson VP. Are NBME examination scores useful in selecting radiology residency candidates? Acad Radiol, 2000, 7, 603-606. 3. Clarke JR, Wigton RS. Development of an objective rating system for residency applications. Surgery, 984, 96, 302–307. 4. Komives E, Weiss ST, Rosa RM. The applicant interview as a predictor of resident performance. J Med Educ, 984, 59, 425– 426. 5. Smith WL, Wood PS, Altmaier EM, O’Halloran C, Franken EA Jr. Critical performance behaviors of radiology residents: Evaluation of two category systems. Am J Roentgenol, 992, 59, 885–888. 6. Altmaier E, Smith WL, Wood P, Ross R, Montgomery W, Klatte E, et al. Cross-institutional stability of behavioral crite- ria desirable for success in radiology residency. Invest Radiol, 989, 24, 249–25. L A M K I E T A L Method Definitions of the Attributes Selected 1. Interpersonal skills 2. Recognition of limits 3.  Curiosity 4. Conscientiousness 5. Confidence Sample Questions Asked at the Interview 1. Interpersonal skills 2. Recognition of limits 3. Curiosity 4. Conscientiousness 5. Confidence Data analysis and selection process Discussion References