Department of Pharmacology, Gujarat Medical Education & Research Society Medical College, Gotri, Vadodara, Gujurat, India *Corresponding Author e-mail: dr.tkp2006@yahoo.co.in إدراك استخدام املخطط التفصيلي للتقييم النظري التشكيلي يف تدريس علم األدوية تيج��س ب�تيل, م�نوج كوم�ر �شورب�ه, براف�تي ب�تيل abstract: Objectives: This study aimed to assess perceptions of the use of a blueprint in a pharmacology formative theory assessment. Methods: This study took place from October 2015 to February 2016 at a medical college in Gujurat, India. Faculty from the Department of Pharmacology used an internal syllabus to prepare an assessment blueprint. A total of 12 faculty members prepared learning objectives and categorised cognitive domain levels by consensus. Learning objectives were scored according to clinical importance and marks were distributed according to proportional weighting. A three-dimensional test specification table of syllabus content, assessment tools and cognitive domains was prepared. Based on this table, a theory paper was created and administered to 126 pharmacology students. Feedback was then collected from the faculty members and students using a 5-point Likert scale. Results: The majority of faculty members agreed that using a blueprint ensured proper weighting of marks for important topics (90.00%), aligned questions with learning objectives (80.00%), distributed questions according to clinical importance (100.00%) and minimised inter-examiner variations in selecting questions (90.00%). Few faculty members believed that use of the blueprint created too many easy questions (10.00%) or too many difficult questions (10.00%). Most students felt that the paper had a uniform distribution of questions from the syllabus (90.24%), that important topics were appropriately weighted (77.23%), was well organised (79.67%) and tested in- depth subject knowledge (74.80%). Conclusion: These findings indicate that blueprinting should be an integral part of written assessments in pharmacology education. Keywords: Medical Education; Educational Measurement; Perceptions; Students; Faculty; Reliability and Validity. الأدوية. علم يف الت�شكيلي النظري للتقييم التف�شيلي املخطط ا�شتخدام اإدراك تقييم اإىل الدرا�شة هذة تهدف الهدف: امللخ�ص: الطريقة: اأجريت هذة الدرا�شة من اأكتوبر 2015 اإىل فرباير 2016 يف كلية الطب بجوجرات, الهند. ق�م اأع�ش�ء هيئة التدري�س بق�شم علم الأدوية ب��شتخدام املنهج الداخلي لتح�شري التقييم للمخطط التف�شيلي. اأعد 12 من اأع�ش�ء هيئة التدري�س اأهداف التعليم ومت ت�شنيف م�شتوي�ت جم�لت املعرفة ب�لإجم�ع. مت ت�شجيل اأهداف التعليم وفق� لأهميته� ال�رسيرية و كذلك مت توزيع الدرج�ت ح�شب الرتجيح الن�شبي. مت اأعداد اأختب�ر ثالثي الأبع�د بن�ءا على ق�ئمة حمتوي�ت املنهج, اأدوات التقييم, واملج�لت املعرفية. بن�ءا على هذة الق�ئمة, مت اأنت�ج ورقة نظرية و تقدميه� لعدد 126 ط�لب� يف علم الأدوية. مت جمع ردود الأفع�ل من هيئة التدري�س و الطلبة ب��شتخدام مقي��س ليكرت ذو اخلم�س نق�ط. النتائج: الغ�لبية العظمى من اأع�ش�ء هيئة التدري�س اأتفقوا على اأن اأ�شتخدام املخطط التف�شيلي يكفل التوازن ال�شحيح للدرج�ت ب�لن�شبة للموا�شيع اله�مة )%90.00(, مت��شي الأ�شئلة مع اأهداف التعليم )%80.00(, توزيع الأ�شئلة ح�شب الأهمية ال�رسيرية )%100.00( و التقليل من الأختالف�ت بني املمتحنني يف اإختي�ر الأ�شئلة )%90.00(. اأعتقد القليل من اأع�ش�ء هيئة التدري�س اأن ا�شتخدام املخطط التف�شيلي خلق العديد من الأ�شئلة ال�شهلة )%10.00( اأو الكثري من الأ�شئلة ال�شعبة )%10.00(. �شعر معظم الطلبة اأن الورقة �شملت توزيع متم�ثل لالأ�شئلة من املنهج )%90.24(, املوا�شيع املهمة ك�نت ذات توازن جيد )%77.23(, وك�نت منظمة )%79.67( و اأختربت املعرفة املتعمقة للموا�شيع )%74.80(. اخلال�صة: ت�شري هذة النت�ئج اإىل �رسورة اأن يكون املخطط التف�شيلي جزءا ل يتجزاأ من التقييم الكت�بي يف تدري�س علم الأدوية. الكلمات املفتاحية: التعليم الطبي؛ قي��س التعليم؛ الإدراك؛ الطلبة؛ هيئة التدري�س؛ املوثوقية و امل�شداقية. Perceptions of the Use of Blueprinting in a Formative Theory Assessment in Pharmacology Education *Tejas Patel, Manoj K. Saurabh, Parvati Patel clinical & basic research Sultan Qaboos University Med J, November 2016, Vol. 16, Iss. 4, pp. e475–481, Epub. 30 Nov 16 Submitted 25 Apr 16 Revisions Req. 12 Jun & 31 Jul 16; Revisions Recd. 9 Jul & 6 Aug 16 Accepted 11 Aug 16 doi: 10.18295/squmj.2016.16.04.012 Advances in Knowledge - This study prepared a blueprint for a formative theory assessment in pharmacology. A blueprint aligns an assessment with learning objectives and distributes questions according to weighting based on clinical importance and the core learning objectives of the syllabus. - The findings of this study indicated that the majority of the faculty members and students had positive perceptions of the use of blueprinting in the formation of the assessment. Application to Patient Care - A blueprint is an important approach to stimulating deep learning among medical students, thus indirectly influencing future patient care and clinical practice. - Positive attitudes towards scientific research among undergraduate medical students is likely to enhance the quality of future patient care. Perceptions of the Use of Blueprinting in a Formative Theory Assessment in Pharmacology Education e476 | SQU Medical Journal, November 2016, Volume 16, Issue 4 Written examinations are the most commonly employed method to assess knowledge in medical education and are used to test recall abilities as well as higher-order cognitive functions, such as the interpretation of data and problem-solving skills.1,2 Valid assessment methods are necessary to determine whether students have learned the required information.3 Content validity gauges the extent to which an assessment covers a representative sample of the material which should be assessed; for example, if examination questions cover the learning objectives of the syllabus, the examination is considered to have content validity.4 In contrast, construct validity covers all aspects of subject knowledge such as application, data gathering and interpretation as a collection of interrelated components, which together allow for the assessment to make sense.5 Construct under-representation and construct irrelevance variance are two important challenges to construct validity. If an examiner creates a test with overly easy or difficult questions, asks well-known questions or uses unnecessarily complex language, it would mean that there is construct irrelevance variance, which can lead to the inflation or deflation of test scores.6,7 Content under-representation in a paper can occur due to the inadequate weighting of marks for clinically relevant topics, unequal distribution of course content across the assessment or examiner bias, such as a tendency to focus on popular topics.7 Moreover, the teachers who deliver lectures are usually not the ones who create the assessments; this can reduce the content and construct validity of an assessment, which in turn can lead to a mistrust in the assessment system on the part of the students.7,8 Content imbalance may also result in students focus- ing less on key areas of learning during revision.7 For an assessment paper to be valid, it should match course content, have proportional weighting of content according to clinical importance, consist of questions which are neither overly difficult nor easy and have multiple tools to determine various types of information.7 Blueprinting can be defined as the creation of a template to determine the content of a test; it lists the number and type of questions across the course content, with learning objectives and relative weight- ing given to each topic.7,8 A blueprint provides a systematic multi-step approach to an assessment, defining the purpose (e.g. formative/summative and written/practical) and scope (e.g. for undergraduate or postgraduate students) of the test in order to subsequently determine content and method of assessment. Based on the content, learning object- ives and their domains are identified and differ- ent assessment tools are chosen, such as short answer questions (SAQs), essay questions (EQs) or multiple choice questions.7,8 The content of the assessment is then proportionally weighted as per clinical importance, learning domains and methods of assessment. The total weighting of the number of items to be included in the assessment is decided and a three-dimensional (3D) table of test specifications is prepared to align content, learning domains and assessment tools, as well as prepare individual questions.7,8 Blueprinting is increasingly used in the field of medical education worldwide.9,10 In the UK, assessments created using a blueprint are considered essential to enable future doctors to meet mandatory standards; the assessments are prepared in such a way that students who have not met important learning outcomes are not able to graduate.10 Previous research indicates that blueprinting optimises student assessment, making a positive impact and helping them to focus on key areas in an examination, thereby improving performance.11–13 This approach can reduce inter-individual variability by providing guidance to examiners; moreover, set question papers are usually more valid and reliable than those created without a blueprint.7 However, studies conducted on the use of blueprinting in India are scarce. At the Gujarat Medical Education & Research Society (GMERS) Medical College in Gotri, Vadodara, Gujurat, India, a five-year undergraduate medical education programme results in a Bachelor of Medicine and Bachelor of Surgery (MBBS). Pharmacology education is mainly taught to second-year MBBS students for a total of three semesters of six months’ duration each. The assessment system consists of one summative and two formative examinations, with two written papers in the summative examination and one written paper in one of the formative examinations. Each paper is scored out of 58 marks, although students now receive up to 40 marks due to optional questions. The papers include constructed-response open-ended questions including EQs (4 marks each), short EQs (3 marks each) and SAQs (2 marks each); the maximum number of marks for EQs, short EQs and SAQs are 20, 24 and 14, respectively. In order to pass, a student must get at least 50% on the examination. Traditionally, the assessment content is determined by a paper setter who selects the questions according to the syllabus and question format. This study aimed to prepare a blueprint for a written theory paper and to analyse subsequent feedback on its use in a formative examination by pharmacology faculty members and students. Tejas Patel, Manoj K. Saurabh and Parvati Patel Clinical and Basic Research | e477 Methods This study was conducted between October 2015 and February 2016 at the Department of Pharmacology of GMERS Medical College. In order to familiarise departmental faculty members with the concept of preparing a theory paper via blueprinting, a pilot test consisting of 16 SAQs worth two marks each was administered to 12 members of the faculty. A half- day interactive session was then conducted in three sessions: the first focused on assessment methods and tools, the second on validity and reliability in assessments and the third on the purpose and implementation of blueprinting, including the weighting of topics and assessment methods and how to prepare tables of test specifications. Faculty members were then re-tested using the same SAQs in order to determine the learning outcomes of the interactive session. The syllabus of the first formative theory examina- tion for second-year MBBS students was used to prepare a blueprint for the written paper, consisting of seven topics: general pharmacology; the autonomic nervous system; the peripheral nervous system; the respiratory system; the gastrointestinal tract; auta- coid-related drugs; and drugs affecting blood and blood formation. A literature review of the under- graduate regulations, vision documents, essential drug lists and national health programmes of the Medical Council of India as well as pharmacology textbooks and previous test papers was undertaken to determine learning objectives for each topic.14–20 Each learning objective was categorised into either recall or reason- ing domain categories as per Miller’s pyramid of com- petence.21 Learning objectives and cognitive domain categorisations were then discussed by the same 12 faculty members who had participated in the pilot test. Each faculty member scored learning objectives individually according to clinical importance, with a score of 3 indicating high importance, a score of 2 indicating moderate importance and a score of 1 reflecting little/no clinical importance.7 Mean scores for each learning objective were calculated using a Microsoft Excel spreadsheet, Version 2010 (Microsoft Corp., Redmond, Washing- ton, USA). Differences were resolved via consensus. Based on the learning objective scores, total scores were calculated for each topic as well as the overall syllabus. Proportional weighting was calculated for each topic by dividing the topic score with the total syllabus score.3,7 Table 1 shows the learning objectives, total scores, proportional weighting and mark distribution calculations. Based on the literature review and faculty consensus, a total of 292 learning objectives were identified. The total syllabus score was 663 and the maximum number of marks on the theory test paper was 58, with marks in each topic weighted proportionally. In the next phase, a 3D table of test specifications for the distribution of marks was prepared by aligning content, assessment tools and cognitive domain categories. Table 2 shows the 3D table of test specifications, representing the mark distribution for each topic according to assessment tool (e.g. EQ, short EQ or SAQ) and cognitive domain category (i.e. recall or reasoning). In terms of assessment tools, EQs were used only to assess reasoning abilities (20 marks; 100.00%). Short EQs were more often used to assess reasoning (15 marks; 62.50%) rather than Table 1: Learning objectives, mean total scores, proportional weighting and distribution of marks per syllabus topic Syllabus topic Learning objectives Mean total score* Proportional weighting† Distribution of marks‡ General pharmacology 87 201 0.30 17.40 (17) Autonomic nervous system 70 163 0.25 14.50 (15) Peripheral nervous system 24 45 0.07 4.06 (4) Gastrointestinal tract 21 44 0.07 4.06 (4) Respiratory tract 17 43 0.06 3.48 (4) Autacoid-related drugs 31 71 0.11 6.38 (6) Drugs affecting blood and blood formation 42 96 0.14 8.12 (8) Total 292 663 1.00 58.00 (58) *Learning objectives were scored by faculty members according to clinical importance, with a score of 3 indicating high importance, a score of 2 indicating moderate importance and a score of 1 reflecting little/no clinical importance. †Proportional weighting was calculated by dividing the mean total score per topic by the total syllabus score. ‡Distribution of marks was determined by multiplying the proportional weighting by the total number of marks overall. The numbers within the brackets show the actual marks allotted per topic in the test paper. Perceptions of the Use of Blueprinting in a Formative Theory Assessment in Pharmacology Education e478 | SQU Medical Journal, November 2016, Volume 16, Issue 4 recall (9 marks; 37.50%). In comparison, SAQs were more frequently used to test recall (8 marks; 57.14%) rather than reasoning (6 marks; 42.86%). Overall, the distribution of recall (17 marks; 29.31%) to reasoning (41 marks; 70.69%) marks was approximately 30:70. Based on the blueprint, a paper setter prepared individual test questions following good practices.7 The questions were framed using directive verbs from the revised Bloom’s taxonomy of learning domains to assess the appropriate cognitive domain according to the blueprint and to give students clear directions for their responses.22 The verb “remember” was used to frame recall questions, while the verbs “understand”, “apply” and “analyse” were used for reasoning questions.22 The verbs “evaluate” and “create” were not used. The paper setter selected questions of high, moderate and no/little clinical importance in a ratio of 60:30:10. The paper was then administered to a total of 126 second year pharmacology students. The 12 faculty members were requested to provide feedback regarding their perceptions of the quality and use of the theory assessment paper designed using blueprinting. They were provided with two previous assessment papers for comparison. Feedback was collected using closed- and open-ended questionnaires. For the former, participants were given 14 statements and asked to respond on a 5-point Likert scale using the following responses: strongly disagree, disagree, neutral, agree and strongly agree. In the open-ended questionnaire, they were asked to describe benefits and difficulties in the preparation and implementation of the blueprint as well as the feasibility of this approach for future implementation in formative assessments. The perceptions of the students regarding the paper were determined one week after the assessment. A questionnaire consisting of nine statements was distributed and the students were asked to respond on the previously described 5-point Likert scale. For the pilot test, the SAQ scores of the faculty members were presented as means and standard deviations and compared using a paired t-test. Results from the feedback questionnaires were presented as percentages. Strongly disagree and disagree responses were merged, as were the agree and strongly agree responses. Statistical analysis was performed using GraphPad Prism, Version 6.0 (GraphPad Software Inc., La Jolla, California, USA). A P value of <0.050 was considered statistically significant. This study received ethical approval from the Institutional Human Ethics Committee of GMERS Medical College, Gotri (IHEC #101/2015/ Pharmacology-16). Informed consent was obtained from the faculty and undergraduate medical students before data collection. Results Among the 12 faculty members, the mean test scores before and after the interactive session were 7.72 ± 4.19 and 19.29 ± 5.29, respectively, out of a maximum score of 32 (paired t-test t-value: 10.23; degree of freedom: 11; P <0.001). In total, 10 out of 12 faculty members gave their opinions of the blueprint (response rate: 83.33%). The two remaining faculty members were transferred before the formative assessment was conducted. All of the faculty members agreed that the blueprint ensured a uniform distribution of questions across the syllabus topics, helped to maintain a balance between questions in the recall and reasoning domains and assured the distribution of questions according to clinical importance. In addition, the majority agreed Table 2: Table of test specifications showing the distribution of marks for each topic according to assessment tool and cognitive domain category Syllabus topic Marks allotted (questions) EQs Short EQs SAQs Total Reasoning Recall Reasoning Recall Reasoning General pharmacology 8 (2) 3 (1) 3 (1) 2 (1) 2 (1) 18 (6) Autonomic nervous system 4 (1) 3 (1) 3 (1) 2 (1) 2 (1) 14 (5) Peripheral nervous system - - 3 (1) - - 3 (1) Gastro-intestinal tract - - 3 (1) 2 (1) - 5 (2) Respiratory system 4 (1) - - - - 4 (1) Autacoid-related drugs 4 (1) - - 2 (1) - 6 (2) Drugs affecting blood and blood formation - 3 (1) 3 (1) - 2 (1) 8 (3) Total 20 (5) 9 (3) 15 (5) 8 (4) 6 (3) 58 (20) EQs = essay questions; SAQs = short answer questions. Tejas Patel, Manoj K. Saurabh and Parvati Patel Clinical and Basic Research | e479 that the blueprint resulted in the adequate weighting of important topics (90.00%), aligned questions with learning objectives (80.00%), ensured well-organised theory test papers (70.00%), tested in-depth subject knowledge (60.00%) and minimised inter-examiner variations in selecting questions (90.00%). Few faculty members believed that the blueprint created too many easy questions (10.00%) or too many difficult questions (10.00%) [Figure 1]. The majority of faculty members (90.00%) believed that a blueprint should be incorporated into the design of future examinations and all of them agreed that the blueprint should be an integral part of theory paper framing. Moreover, the majority of faculty members thought that blueprints should be prepared for the entire syllabus (90.00%) and that there was a need to change teaching schedules as per the blueprint (90.00%) [Figure 2]. Table 3 summarises faculty responses to open-ended questions about their perceptions of the blueprint and assessment paper. A total of 123 out of 126 students provided feed- back regarding the formative assessment paper (res- ponse rate: 97.62%). The majority of students believed that the distribution of questions was uniform and covered each topic (90.24%) and allowed for proper weighting of clinically important topics (77.23%). In addition, the paper was generally perceived to be well organised (79.67%). Most of the students believed that all of the questions were from the defined syllabus (90.24%) and that the paper tested in-depth knowledge of the subject (74.80%). Overall, few students believed that there were too many easy questions (12.20%) or too many difficult questions (9.75%); indeed, only a minority of the students felt that the paper was exhaust- ing/lengthy (26.83%) or stressful (17.08%) [Figure 3]. Discussion An assessment drives, directs and influences learning and is a tool for educational improvement.7,23 Students successfully learn when a relationship exists between teaching, assessment and results. In an assessment, every question format (e.g. EQs, short EQs and SAQs) has advantages and disadvantages; using a variety of formats helps to counter the possible bias associated with one individual format.1 A blueprint helps to Figure 1: Perceptions of faculty members regarding the use of a blueprint in the formation of a pharmacology theory assessment paper (N = 10). *One faculty member did not respond to these statements. Figure 2: Perceptions of faculty members regarding the feasibility of future implementation of a blueprint in pharmacology education (N = 10). Table 3: Summary of faculty responses to open-ended questions regarding the use of a blueprint in the formation of a pharmacology theory assessment paper (N = 10) Open-ended responses What are the advantages of using a blueprint? •It makes the assessment more objective •There is less chance of subjectivity, bias and prejudice •It makes the theory test examination fairer •Students are tested on different cognitive domains •It is easy for the examiner to prepare question papers •It creates a well-organised paper and syllabus What are the disadvantages of using a blueprint? •It is time-consuming to score learning objectives by clinical importance •It takes time to adjust marks according to recall and reasoning domains •It is sometimes difficult to reach a consensus between faculty members •It will require periodic revisions in future •The choice of questions is restricted •There is a chance that questions in the paper will be more predictable What were the differences between the paper created with a blueprint and previous years’ papers? •The questions in the blueprint-created paper were more specific •The distribution of marks and questions followed the recommendations of the blueprint •There was greater clarity in student assessment •The blueprint-created paper allowed for more questions in a higher-function cognitive domain •The paper devised with a blueprint was well structured Perceptions of the Use of Blueprinting in a Formative Theory Assessment in Pharmacology Education e480 | SQU Medical Journal, November 2016, Volume 16, Issue 4 ensure the balance between different format questions by specifying the content, learning domains across the syllabus and assessment tools of an examination in a rational manner.7,8 In the current study, a blueprint was used to create a written assessment paper for pharmacology students; learning objectives were identified and syllabus topics were marked using a proportional weighting system based on clinical importance to ensure content validity. Moreover, the paper assessed not only knowledge recall, but also understanding, application and critical analysis abilities using the interrelated cognitive domain of reasoning, thus ensuring construct validity.5 In the current study, the majority of faculty members and students agreed that the use of a blueprint in the formation of a pharmacology forma- tive assessment paper resulted in a uniform distrib- ution of questions and adequate weighting of clinically important topics and testing of in-depth subject knowledge. Faculty members also agreed that the test paper struck an appropriate balance between questions in the recall and reasoning domains and that the questions were aligned with the learning objectives of the syllabus. These findings suggest that the test paper created through the blueprint was able to avoid construct under-representation. Previous research has also indicated that the use of a blueprint in assessment ensures coverage of all aspects of the curriculum, learning objectives and educational domains.5,24,25 Very few faculty members and students in the current study believed that the assessment contained questions which were too easy or too difficult; this suggests that the use of a blueprint resulted in a rational paper and avoided construct irrelevance variance. Moreover, the blueprint seemed to make the assessment more ‘fair’ in the eyes of both faculty members and students. The experience of an authentic assessment seems to be a motivating factor; among medical students, a fair assessment has been reported to stimulate a deep approach to learning when tailored to curricular objectives.26 This may subsequently affect clinical practice and patient care once the students have graduated. Earlier reports in the literature have provided evidence that blueprints make it easier for an exam- iner to select questions, set papers according to accepted norms and standards, test higher-order cognitive domains and create well-organised theory papers.7,24 These findings were also reflected in the current study during faculty responses to open-ended questions concerning the benefits of blueprinting. However, certain challenges to the use of blueprinting were also identified; many faculty members stated that the preparation of a blueprint was time-consuming and that it was occasionally difficult to reach a consensus. Nevertheless, despite these barriers, the majority of faculty members found blueprinting to be a promising approach for preparing assessment papers and indicated that they would use this approach in future examinations and change their teaching patterns accordingly. Overall, the use of a blueprint in a formative assessment paper had a positive impact on faculty perceptions; it is therefore realistic to recommend the inclusion of this approach as a valid tool to frame theory papers for summative assessment. Nevertheless, the data from this study would be more valuable if they were combined with evidence of increased academic achievement due to the blueprint; unfortunately, the study was limited in this regard as the students’ performance could not be compared to students in previous years due to the differences in syllabus. Conclusion According to the perceptions of both pharmacology faculty members and students, the use of blueprinting in the formation of a written formative assessment paper was found to result in high content and construct validity. As this approach helps to align the content, cognitive domains and assessment tools of a paper in a rational way, it should be implemented as an integral part of the framing of theory assessments in future. c o n f l i c t o f i n t e r e s t The authors declare no conflicts of interest. f u n d i n g No funding was received for this study. Figure 3: Perceptions of students regarding the use of a blueprint in the formation of a pharmacology theory assessment paper (N = 123). Tejas Patel, Manoj K. Saurabh and Parvati Patel Clinical and Basic Research | e481 a c k n o w l e d g e m e n t s The authors wish to thank all of the faculty members and students who participated in this study. In addition, they are grateful to Dr. Neena Doshi of the GMERS Medical College, Gotri, for critically reviewing the manuscript before publication. References 1. Al-Wardy NM. Assessment methods in undergraduate medical education. Sultan Qaboos Univ Med J 2010; 10:203–9. 2. Ghosh A, Mandal A, Das N, Tripathi SK, Biswas A, Bera T. Students’ performance in written and viva-voce components of final summative pharmacology examination in MBBS curriculum: A critical insight. Indian J Pharmacol 2012; 44:274–75. doi: 10.4103/0253-7613.93870. 3. Tabish SA. Assessment methods in medical education. Int J Health Sci (Qassim) 2008; 2:3–7. 4. Alias M. Assessment of learning outcomes: Validity and reliability of classroom tests. World Trans Eng Technol Edu 2005; 4:235–8. 5. Adkoli BV. Attributes of a good question paper. In: Sood R, Ed. Assessment in Medical Education: Trends and Tools, 1st ed. New Delhi, India: KL Wig Centre for Medical Education & Technology, 1995. Pp. 65–82. 6. Downing SM. Threats to the validity of locally developed multiple-choice tests in medical education: Construct- irrelevant variance and construct underrepresentation. Adv Health Sci Educ Theory Pract 2002; 7:235–41. doi: 10.1023/ A:1021112514626. 7. Adkoli BV, Deepak KK. Blueprinting in assessment. In: Singh T, Anshu, Eds. Principles of Assessment in Medical Education, 1st ed. New Delhi, India: Jaypee Brothers Medical Publishers Ltd., 2012. Pp. 205–13. doi: 10.5005/jp/books/11647_19. 8. Patil SY, Gosavi M, Bannur HB, Ratnakar A. Blueprinting in assessment: A tool to increase the validity of undergraduate written examinations in pathology. Int J Appl Basic Med Res 2015; 5:S76–9. doi: 10.4103/2229-516X.162286. 9. Ten Cate O. Medical education in the Netherlands. Med Teach 2007; 29:752–7. doi: 10.1080/01421590701724741. 10. General Medical Council. Assessment in undergraduate medical education. From: www.gmc-uk.org/Assessment_in_ undergraduate_me dical_e ducation___guidance_under_ review_0815.pdf_56439668.pdf Accessed: Aug 2016. 11. Mookherjee S, Chang A, Boscardin CK, Hauer KE. How to develop a competency-based examination blueprint for longitudinal standardized patient clinical skills assessments. Med Teach 2013; 35:883–90. doi: 10.3109/0142159X.2013.809408. 12. Ahmad RG, Hamed OA. Impact of adopting a newly developed blueprinting method and relating it to item analysis on students’ performance. Med Teach 2014; 36:S55–61. doi: 10.31 09/0142159X.2014.886014. 13. McLaughlin K, Coderre S, Woloschuk W, Mandin H. Does blueprint publication affect students’ perception of validity of the evaluation process? Adv Health Sci Educ Theory Pract 2005; 10:15–22. doi: 10.1007/s10459-004-8740-x. 14. Medical Council of India. Regulations on graduate medical education, 1997. From: www.mciindia.org/Rules-and-Regu lation/GME_REGULATIONS.pdf Accessed: Aug 2016. 15. Medical Council of India. Vision 2015. From: www.mciindia. org/tools/announcement/MCI_booklet.pdf Accessed: Aug 2016. 16. Central Drugs Standard Control Organization. Report of the Core-Committee for Revision of National List of Essential Medicines: November 2015. From: cdsco.nic.in/ Wr i t e R e a d D a t a / N L E M - 2 0 1 5 / R e c o m m e n d a t i o n s . p d f Accessed: Aug 2016. 17. Katzung BG, Masters SB, Trevor AJ. Basic and Clinical Pharmacology, 12th ed. New York, USA: McGraw-Hill Medical, 2012. Pp. 1–1170. 18. Tripathi KD. Essentials of Medical Pharmacology, 7th ed. New Delhi, India: Jaypee Brothers Medical Publishers Ltd., 2013. Pp. 1–973. 19. Satoskar RS, Rege NN, Bhandarkar SD. Pharmacology and Pharmacotherapeutics, 24th ed. Guragon, India: Elsevier Health INR, 2015. Pp. 1–1127. 20. Sharma HL, Sharma KK. Principles of Pharmacology, 2nd ed. Hyderabad, India: Paras Publications, 2012. Pp. 1–928. 21. Miller GE. The assessment of clinical skills/competence/ performance. Acad Med 1990; 65:S63–7. doi: 10.1097/00001888- 199009000-00045. 22. Adams NE. Bloom’s taxonomy of cognitive learning objectives. J Med Libr Assoc 2015; 103:152–3. doi: 10.3163/1536-5050. 103.3.010. 23. Astin AW, Banta TW, Cross KP, El-Khawas E, Ewell PT, Hutchings P, et al. American Association for Higher Education (AAHE) principles of good practice for assessing student learning. From: www.algonquincollege.com/profres/ files/2013/10/9-Principles-of-Good-Practice-for-Assessing- Student-Learning.pdf Accessed: Aug 2016. 24. Patil SY, Hashilkar NK, Hungund BR. Blueprinting in assessment: How much is imprinted in our practice? J Educ Res Med Teach 2014; 2:4–6. 25. Coderre S, Woloschuk W, McLaughlin K. Twelve tips for blueprinting. Med Teach 2009; 31:322–4. doi: 10.1080/ 01421590802225770. 26. Al Kadri HM, Al-Moamary MS, Magzoub ME, Roberts C, van der Vleuten C. Students’ perceptions of the impact of assessment on approaches to learning: A comparison between two medical schools with similar curricula. Int J Med Educ 2011; 2:44–52. doi: 10.5116/ijme.4ddb.fc11.