"Viva Voce": Oral Exams as a Teaching & Learning Experience “VIVA VOCE”: ORAL EXAMS AS A TEACHING & LEARNING EXPERIENCE Peter M. Markulis SUNY Geneseo Markulis@Geneseo.Edu Daniel R. Strang SUNY Geneseo ABSTRACT This paper describes the background and use of the oral exam format in a strategic management course. The paper discusses the history and rationale for using oral exams, and provides a model for instructors of upper level business courses to augment their course pedagogy by using the oral exam format. The paper concludes by offering a set of guidelines for implementing the oral exam methodology in selected upper-level business courses. INTRODUCTION Viva Voce is a Latin expression that was used in Catholic seminary education to refer to oral exams (it literally means, “living voice”). By tradition, oral exams were used not as a substitute, but as a complement to written exams. They are a way to ask what is not feasible through the written format. Ostensibly, the rationale was that instructors could use the oral format to probe, challenge, and critically assess what a student really knew about a particular topic. One seminary instructor fondly stated, “I could tell in less than 5 minutes if a student really knew anything about the subject in question.” It is this perspective which provides a partial rationale for the oral exam format in the strategic management course. For the past six semesters, we have been using an oral exam format as part of the examination process for the senior-level strategic management course. This paper provides the background, rationale, methodology, and results of using that format. The paper also provides a set of guidelines and caveats for using the oral exam format. LITERATURE REVIEW Oral exams are not new, but surprisingly, they are not pervasive. A search of the academic literature elicited virtually nothing in terms of the rationale, use, or design of the oral exam format. Tuft (2006) in the University of Alberta’s official website provides 2 pages Tips for Oral Exams on its help page. Some disciplines, including some of the natural sciences, reported the use of oral exams. For example, Tewksbury, while not arguing explicitly for an oral exam, highly recommends the use of oral-related assignments in the field of geology (1996). This stands to reason as many disciplines, like medicine or dentistry require a “show-and-tell” display before one can practice the art and/or science. The University of Waterloo’s Newsletter (of 2004) features a short paragraph on Professor Hannah Wilson’s use of the interactive midterm oral in one of her geography classes. Wilson offers this comment from a student on the use of the interactive midterm oral exam format, “The interaction in the exam helps in learning and consolidating as you think of how to answer instead of regurgitating information.” Bridges makes an argument for using the oral exam, but her proposal is confined to case exams for marketing courses (1999). She provides the following rationale for using oral exams in marketing courses: • Provides the students with the opportunity to develop and demonstrate oral communication ability; • Give students experience with the communications identified as most challenging in the workplace, i.e., interaction with a superior; • Help students develop explanatory skills, powers of persuasion, oral poise and self-confidence. A more theoretically-grounded model for oral exams usage is offered by Mandeville and Menchaca, although operationalization of their model is limited to group oral exams for selected teacher-education courses. They support their rationale by drawing upon the social collectivist theory of learning (Vygotsky, 1978), which argues that thinking abilities develop as a result of attempting to communicate thoughts and positions as one matures. Knowledge is first externally constructed among knowledge members of a community and then internally constructed by each member of the community. A key to understanding this concept is that “language and dialogue are critical to the development of knowledge…” The authors posit that part of the learning process is accomplished through the interchange between instructor and student, which takes place through the dialogic of the oral exam. It is important to note that this model explicitly recognizes learning takes place in the oral exam setting. Most doctoral programs use the comprehensive oral exam—or exams—as a prelude to entry into the dissertation phase of the program. Oral exams are a way, one assumes, for prospective Ph.D. students to relate what they know but Developments in Business Simulation and Experiential Learning, Volume 35, 2008 118 mailto:Markulis@Geneseo.Edu more importantly, are a way in which doctoral mentors can assess the full range of Bloom’s cognitive domains to see if prospective Ph.D. students really know their material (Bloom, et al. 1956). RATIONALE It is interesting to reflect on why oral exams have not been used (more?) in business schools as a standard part of the examination and certification process. A recent study investigating the views of administrators toward the inclusion of comprehensive examinations as part of a master of business administration (MBA) degree program found little support for the oral exam format. Administrators at sixty mid-western institutions were surveyed as to their use of comprehensive exams for MBA students. Forty-seven responses were received. Of interest is that only fifteen institutions used comprehensive examinations, and of that, two-thirds had a written exam format and while only 20% used an oral exam format. The study reports that many institutions dropped the comprehensive format citing issues of administrative problems, educational worth, and market competition. Given the expectations of MBA graduates in terms of oral communications in the business profession, this result is surprising. The AACSB (Association to Advance Collegiate Schools of Business) stresses the importance of developing oral communication skills (2005). The AACSB requirement that participating schools have a set of educational objectives for each program and that there be required coverage of certain proficiencies, indicate the a clear movement within traditional business education toward ensuring business students possess needed skills. Even though AACSB explicitly specifies written and oral communication skills, they do not provide content examples or measurable standards that indicate individuals’ ability to apply the relevant skills outside the classroom. AACSB’s mission-driven accreditation guidelines suggest soliciting stakeholders’ input and the subsequent incorporation of this feedback into program improvements. Frequently this involves a process of surveying employers, recruiters, and/or alumni requesting them to rank or rate lists of skills as to their importance and the school’s performance on these skills. a. The following is a “typical” list of oral communications skills students are expected to learn (or experience in) in an undergraduate business curriculum. b. deliver self-prepared speeches in a public setting; c. demonstrate the basic principles for organizing ideas appropriately for accomplishing informative and persuasive communication objectives; d. understand and demonstrate the principles of audience-centered message adaptation; e. locate, use, and correctly cite appropriate evidence in supporting their claims; f. demonstrate communication behaviors appropriate for effective comprehensive and supportive listening; g. understand and be able to apply the communication behaviors appropriate for the constructive management of interpersonal and intragroup conflict; h. understand the skills, roles, and methods of proceeding in task groups in order to achieve high levels of motivation, productivity, and member satisfaction and to obtain high-quality decisions and/or outputs; and i. understand the components of the communication process and how they enhance and/or hinder the effective exchange of information and ideas. As noted above, some instructors have used—or, suggest the use of--oral exams in specialty areas or in specialized circumstances. However, other than Mandeville and Menchaca’s work, there seems to be little in the way of a model, or rationale for their use. Key Questions: This leaves one with two important questions: • what type(s) of oral communications actually occur(s) in the world of business? • what learning theory or theories support the use of oral exams? The first question concerns what students ought to know—or, at least have some practical experience in— when they enter the world of business. What are the types of oral communications they will experience in the world of business? Specifically, what should business schools be teaching undergraduate students and exposing them to, in terms of oral business communications skills? By oral communication skills, many business schools mean the formal, oral presentation, whereby a student (or team of students) stands before an audience (usually their classmates and a faculty member) and makes a polished PowerPointTM presentation on some subject. The relevant question, of course, is: will formal, oral presentations--often the choice of business school teaching—be the venue most often experienced by business school graduates when they enter the workforce? Evidence indicates that the formal presentation is but one of several important ways in which oral communications take place in the business world. Volkema and Niederman suggest that managers (and, one assumes, prospective managers), need to be able to express themselves in both formal and informal ways for the purposes of : • training; • problem-solving; • monitoring and coordinating organizational situations; • delegating; and, • information exchanges (1996, p. 275-6). The following is a three-dimensional figure, which illustrates a set of “typical” oral communication interactions one would expect to observe in a business setting: Developments in Business Simulation and Experiential Learning, Volume 35, 2008 119 FIGURE 1 The Set of “Typical” Oral Communication Interactions Modalities Purposes K now /Fact G athering C larity C ritique/C hallenge Persuade A nalyze/Synthesize Inform Constituencies Superiors Peers Subordinates Outside Constituencies Formal Team Informal Team One-on-One Formal Meeting One-on-One Informal As can be seen from the above figure, formal presentations represent only one of many oral communication venues. Further, much organizational work is conducted by and through teams. The ability to effectively and efficiently discuss, present, clarify and analyze organizational issues is most frequently done via oral communications when teams are involved. We are unaware of any business school program, which provides specific development in this area. The second question asks: Is there a paradigm, theory, model or rationale that advocates for the oral exam format? Three models or frameworks are instructive in this respect. As briefly mentioned above, Social Constructivism Theory comes from the work of psychologist Lev Vygotsky (1978). While Vygotsky’s model deals principally with the development of knowledge from childhood to adulthood, his theory is helpful in understanding how the oral exam format adds to the articulation abilities of business school students. In his theory, knowledge is assumed first to be externally constructed among members of the community, and then internally constructed by each member of the community. In social constructivism, language and dialogue are critical to the development of knowledge, for it is through dialogue that the community is able to construct common language, which means a similar way in which people can share their understanding, analysis and critical assessments of issues in a common way. It does not mean that there will be common agreement! It is important to understand the internal aspect here. One’s internal construction is shaped and re-shaped through a continual process of constructive dialogue. It is through internal language that individuals construct their idiosyncratic expression of the community’s knowledge. But how does one know if the internal construction is really an understanding or just an opinion without any reasonable or factual basis. We are talking here about ways in which we can share with one another, without necessarily agreeing or reaching the same conclusion. The expression of one’s internal language or reconstructions of the outer world, are important to be expressed if one is to be a valuable contributor to society—or, just a reasonable citizen. This internal construction, or perhaps, reconstruction, is similar to the process McNeil (1987) calls “elaboration.” Clearly, the oral exam format is one in which students are not only called upon to express their “internal constructions,” but to engage in a dialogue which pits their understanding against that of the professor’s. Perhaps the main point here is that one’s internal and external understanding are in a continual state of growth and development due to the dynamics of the oral exam dialogue. A second framework that may be used to support the oral exam mode is the concept of the Discourse Community, as offered by Faigley, (1985). The “Discourse Community” refers to a group joined together by the particular ways in which they use language, for examples, scientist’s jargon, or Developments in Business Simulation and Experiential Learning, Volume 35, 2008 120 the art critic’s choice of metaphor. We are not talking principally about special terms, idiomatic expressions or unique terminology, but about the entire language system that persons in these areas know and use to express the concepts, issues and developments within those areas. Take the language of strategic management, for example. Members of this community know what is worth discussing and they learn the language, metaphors, expressions, intonations, etc., which are best suited for expression of various concepts or the discussion of various issues. They also know how to critique, and analyze issues, and persuade others. In professional organizations, the ability to do this and do it well is an invaluable asset. Knowing and being able to use organizational language legitimizes socio- cultural communications and helps one to feel “in touch.” As Pye (1995) notes, managing is about dialogue and action, and that occurs through listening, talking, creating, shaping, and sharing meaning. This is how things develop in organizational settings. The oral exam methodology clearly provides opportunities for students to experience and develop their skills in this area. A third concept that supports the use of the oral exam format is that of “persuasion.” Eccles and Nohria explain, “Managers live in a rhetorical universe—a universe where language is constantly used not only to communicate but also to persuade and even to create,” (1992, p. 6). Here, we see persuasion as a skill, which demonstrates the application of three abilities • good use of evidence; • reliance on one’s expertise; and, • the ability to use language to evoke an emotional response, (Knights and Morgan, 1991; and Pettigrew, 1985) draw attention to the power of language in shaping ideas by drawing on and controlling the flow of information by… Oral Exam Rationale: Drawing on the above theories and models, oral exams provide a constructive forum in which to: • probe the student’s knowledge and understanding; • challenge the student’s understanding; • ascertain the student’s appropriate use of the ‘language’ of business discourse; • test the student’s persuasive skills, and oral poise. The oral exams format enables instructors to test the students on all five cognitive domains of Bloom’s taxonomy. For example, consider the type of questions and questioning one can use in an oral exam setting. The examiner can ask the student about his/her knowledge and comprehension (levels 1 & 2), can use the exam to see if the student can apply the concepts (level 3), can use a case to test the student’s analytical ability (level 4), can determine if the student can combine concepts into a new whole (Level 5), and can even determine if the student can evaluate or critically assess various concepts or theories (Level 6). While many of these domains can be assessed through the written exam, the oral exam allows the instructor to probe these areas to ascertain if the student “really knows what s/he is talking about.” Oral exams thus cover several cognitive domains, but also the psychomotor skill of oral expression. Again, oral exams are not a substitute for written exams, but another way to determine the depth and breadth of student’s knowledge, understanding and use of various concepts. Consider what one can do on an oral exam as opposed to a written one covering similar issues or concepts. The University XXX Experience. Oral exams have been used for three years in the strategic management course at the Business School of University XXX. The first year, it was implemented on a voluntary basis as a trial. Given the positive feedback, and the view by most faculty members that the format in fact “fit” the school’s educational goals, it was decided to make the oral exam format mandatory. Oral Exam Details. The oral exam format is explained on the course description for the strategic management course. The first day of class, the instructor discusses the details and specifics of how the oral exam works. Students must choose to take one of the exams scheduled for the semester using the oral exam format. For courses scheduled on Tuesdays and Thursdays, there are usually two exams during the semester (not including the case final), while the Monday-Wednesday-Friday schedule lists three exams during the semester (not including the case final). Students are told that there will be sign-up sheets outside the instructor’s office for about a ten-day period (coinciding with the written exam date) listing ½ hour oral exam time slots. In addition students are told that these slots will be limited to about 1/3 of the students taking the course, as the instructor cannot be expected to give all of his/her students registered in the class an oral exam at once. Sign-up are posted for remaining two (or three) exams, so if students miss the first or second oral exam period, they must sign-up for the third one or be subject to a significant penalty. Like many other educational institutions, the School of Business at University XXX has been developing various learning goals for programs and courses, as well as assessment protocols and rubrics. The faculty believed that it would add to the educational value of the oral exam format to have students assist in the development of an oral exam rubric. In order to reasonably accomplish this, the instructors provided to the students several existing rubrics on oral communications, critical thinking, and case analysis, which they could use to help them in developing an oral exam rubric. The assignment was given to teams of students, rather than to individual students, and the teams were given two weeks to submit early in the semester a sample rubric. While many of the rubrics submitted by the student-teams were simplistic, it was clear there were common threads, which could be woven, into a standard oral exam rubric (See APPENDIX A). The course instructors believed that the rubric development process provided a good way to gain student support for the oral exam format, as well as to make them feel more comfortable with the basic idea. Developments in Business Simulation and Experiential Learning, Volume 35, 2008 121 TABLE 1 Selected Survey Results Strongly Agree (or Very High) Agree Indifferent Disagree Strongly Disagree (or Almost None) Q1 19 42 15 21 3 Q2 57 32 5 1 5 Q3 33 35 13 14 5 Q4 16 19 32 18 15 Q9 37 39 15 7 2 Q10 49 38 17 0 6 Q13 25 47 23 4 1 Q17 42 44 8 3 3 Q28 70 20 0 10 0 Q30 76 12 8 2 2 Q1: I studied more strenuously for the oral exam than for traditional exams. Q2: During the oral exam I had adequate time to answer the questions. Q3: What was your Anxiety Level before the oral exam? Q4: What was your Anxiety Level during the oral exam? Q9: The feedback I received at the end of the exam was adequate. Q10: The oral exam was fairly graded. Q13: I feel that the examiner took adequate notes on my performance. Q17: I was given enough time to reflect before responding to the examiner's questions. Q28: Do you think the oral format should be used in selected upper level courses? Q30: Students should be required to take at least one oral exam in at least one course at The School of Business? RESULTS Results should be viewed from both faculty and student perspectives. Drawing on our experience, a set of guidelines and initial considerations is provided for those interested in pursuing the oral exam format. Faculty Involvement. Several faculty members at the Business School were interested in observing the oral exam process in action. Indeed, as word of the oral exam format spread beyond the business school, faculty from other departments expressed an interest in observing the process, as well. The issue of how to allow faculty to observe this process without intimidating students was considered by the instructors. Unfortunately, the University did not have available special rooms for unobtrusive observation purposes, so the instructors had to determine a reasonable approach. To address the potential intimidation issue, course instructors told the students that faculty observers would be present in the room, but their role was to simply observe the process and not participate in the grading. Several faculty members observed the process and passed their comments onto the instructors of the course. Observers included the Dean and Assistant Dean of the Business School as well as Professor XXX, the Director of the University’s Teaching and Learning Center. Student Survey. For the first two semesters in which the oral exam was used as an option for students, no formal student feedback was solicited, although informal comments indicated a favorable view toward the oral exam format. When the faculty decided to make the oral exam format mandatory, it was decided that students’ perceptions be obtained. At the end of each semester, a short survey was given to ascertain student perceptions on the oral exam format. The survey consisted of 32 questions and was used for 4 semesters. Ninety surveys were tabulated. Overall, students felt the experience was a valuable one, and one that should be a standard part of at least one upper level course. A copy of the survey can be found in APPENDIX B, and Table 1 provides selected survey results. Guidelines and Recommendations. Clearly, the oral exam format is not a panacea to be used by all instructors or to be used in every course. Large section classes, professor’s time conflicts, and course goals are just a few of the issues that may mitigate the use of the oral exam format. Below are two sets of guidelines for faculty to consider before choosing the oral exam format. INITIAL CONSIDERATIONS & GUIDELINES Before considering the oral format, instructors should consider the following: • Does the format “fit” with the program/course learning goals? • What added value is there for the oral exam format? • What additional benefit is there to using this format? • Should the focus be Bloom’s Cognitive Domains Developments in Business Simulation and Experiential Learning, Volume 35, 2008 122 • Should the focus be on oral communication skills? • Can/should the format be used in ALL the sections of this course? • What will be the reaction of the students/faculty? • What type of questions will be asked during the oral? • Should the format be used for special or honors- type courses? GUIDELINES AND IMPLEMENTATION ISSUES Before actually using the oral exam format, instructors should address the following: • What should be stated in the course outline/description? • What are my time constraints? • Class size • How will oral exams be scheduled? (remember not to schedule too many at once) • Should the oral exam format be optional or mandatory? • Should the students assist in the development of an oral exam rubric? • Consider length (20-25 minutes) • Consider when and how to provide feedback • Consider the location and seating arrangements • Should the students be surveyed? CONCLUSIONS How many business schools assess oral communication skills solely through formal case presentations? Often the student or student team is expected to make a dazzling formal presentation using PowerPoint slides. Could it be that business schools find the formal PP presentation easy to administer and an easy way in which to cover their oral communication requirement? But as Tufte (2006) notes, asking students to deliver a formal presentation using PowerPoint may actually be a way of discouraging them to think and present information creatively—and often not clearly. For some students, it actually is a way in which to “discourage” critical questioning and analysis. The formal PowerPoint presentation is often a performance, not a venue for challenging, and probing dialogue. The AACSB suggests that: • Development of a portfolio of personal skills that will strengthen their abilities to communicate, solve problems, make decision and lead organizations. • Professional competence within an individual management discipline, the capacity for integrating and applying knowledge from other disciplines, and a strategic perspective on the management of organizations. • Adaptability that comes from the acquisition of knowledge and skills that readily transfer to different work environments and to other dimensions of productive lives. Many of these objectives are hard, if not impossible to measure, through written exams or even through large scale personal portfolios. Clearly, oral exams offer the opportunity to assess student’s progress along several of these dimensions. Oral exams provide a meaningful way in which to test students along all 5 of Bloom’s cognitive domains. Further, they provide a way in which to assess the student’s oral communication abilities. Nonetheless, oral exams cannot be used efficiently in most courses. Issues of program and course goals most be matched with the kinds of skills the program stipulates. Further research is necessary to see what impact oral exams have on several learning factors. Presently, the business school instructors using the oral exam format at University XXX are working in conjunction with the University’s School of Education to design a valid and reliable survey instrument to measure student perceptions, as well as to posit a learning model and determine what meaningful dependent variable or variables are measurable. REFERENCES AACSB International, (2005). “Why Management Education Matters: Its Impact on Individuals, Organizations and Society,” A Report from the AACSB International Task Force of the Committee on Issues in Management Education. Retrieved 7/21/07 from http://www.aacsb.edu/publications/valuereport_lores.pd f . Bridges, S., (1999). Oral Case Exams in Marketing: Enhancing and Evaluating Communication and Problem-Solving Skills. Marketing Education Review, Vol, 9, No. 3, p. 27. Brown, T. & Parker, S. (1989). Comprehensive Exams: Still a Useful Tool? Journal of Professional Services Marketing, New York: Vol. 4, Iss. 2; pg. 101-11. Eccles, R. G. & Nohria, N. (1992). Beyond the Hype: Rediscovering the Essence of Management. Harvard Business School Press, Boston, MA.: p. 6. Faigley, L. (1985). Nonacademic Writing: The Social Perspective. Writing in Non Academic Setting, ed., by L. Odell and D. Goshami, New York: The Guilford Press, pp. 231-248. Forman, J., (1999). Management Communication and MBA education: An Argument for Management Literacy. Selections, Santa Monica, Spring 1999, Vol. 15, Iss. 3, pp. 9-17. Knight, M. (1999). Writing and Other Communication Standards In Undergraduate Business Education: A Study of Current Program Requirements, Practices and Trends. Business Communications Quarterly, Vol. 62, No. 1, March 1999, pp. 10-28. Knights, D. & Morgan, G. (1991). Corporate strategy, organizations and subjectivity: a critique. Organizational Studies, Vol. 12, No. 2, pp. 251-73. Developments in Business Simulation and Experiential Learning, Volume 35, 2008 123 http://www.aacsb.edu/publications/valuereport_lores.pdf http://www.aacsb.edu/publications/valuereport_lores.pdf Mandeville, T. F., & Menchaca, V. (1994). Group Oral Exams: Exploring Assessment Techniques for New Instructional Paradigms. Reading Research & Instruction, Vol. 33 (4), pp. 319-25. Pettigrew, A. (1985). The Awakening Giant. Blackwell Press, Oxford, England. Popham, J., W. (1993). Circumventing the high costs of authentic assessment. Phi Delta Kappan, Vol. 74 (6) pp. 470-4. Pye, A. (1995). Strategy through dialogue and doing. Management Learning, Vol. 26., No. 4, pp. 445-62. Sprayberry, S., L. (1996). Corners, Walls, and Doors: The Methodology of Oral Exams in a Course on American Indian Literature. Studies in American Indian Literature, vol. 8, No. 2, pp. 21-7. Tewksbury, B., J. (1996) Teaching without Exams—The Challenges and Benefits. Journal of Geoscience Education, 44 (4) pp. 366-72. Tufte, E. R. (2006). The Cognitive Style of PowerPoint: Pitching Out Corrupts Within. Sec. Edition. Retrieved from http://www.uofaweb.ualberta.ca/academicsupport/ av03.cfm?nav 3=51996&nav02=50367&nav01=27013 University of Waterloo Newsletter (2004). Tell Me What You Know: The Interactive Oral Midterm. May 2004, N0. 15, page 2. Retrieved from http//www.admim. waterloo.ca/infotrac/tmMay04.html#Strategies) Young, M. R., & Murphy, W. (2003). Integrating Communications Skills into the Marketing Curriculum: A Case Study. Journal of Marketing Education, Vol. 25, Iss., 1, pp. 57-71. Volkema, R. J., & Niederman, F. (1996). Planning and Managing Organizational Meetings: An Empirical Analysis of Written and Oral Communications. The Journal of Business Communications, Vol. 33, N0. 3. July 1996), pp. 275-296. Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press. Developments in Business Simulation and Experiential Learning, Volume 35, 2008 124 http://www.amazon.com/exec/obidos/search-handle-url/105-4903782-8930026?%5Fencoding=UTF8&search-type=ss&index=books&field-author=Edward%20R.%20Tufte APPENDIX A ORAL EXAM RUBRIC Name_____________ Course_____________ Date_________ Trait Unacceptable Acceptable Excellent Score Clarity Unsure & unclear in explanations Responses are basically clear, but simple Responses are quite sophisticated Thoroughness Incomplete in responses Relates basic information or explanation Complete and comprehensive explanations Application of concepts and conceptual material Unable to apply a theory or concept to case, issue or situation Able to apply at least one basic concept or theory Fully applies and creatively uses concepts or theories Understanding of concepts & conceptual material Seems not to understand basic theories or concepts Has basic understanding of concept or theory Can fully explain and expand upon a theory (s) or concept (s) Analysis Cannot seem to break issue or situation into meaningful sub areas Able to segregate and divide issues or situation into meaningful subparts for better understanding of issues Can subdivide issue or situation into meaningful subparts and explain how those subparts add to understanding of issue Perspective Inability to see or appreciate multiple views on issues or situation or understand why they are important Can see and explain the importance of multiple perspectives Without prompting, applies several perspectives to a situation or issues and explains their relevance Evaluation Inability to judge, compare or contrast concept or situation against some external standards of appropriate criteria Simple comparison, judging or contrasting of issue or situation against relevant criteria Can apply evaluative criteria to situation or issue and explain the pros and cons of using such an evaluation method Lessons & Insights Not sure if there are any lessons or insights to be gained from situation or concept Able to relate at least one lesson or insight from the concept or situation Offers several lessons or insights from concepts or situations and explains why they are so Developments in Business Simulation and Experiential Learning, Volume 35, 2008 125 APPENDIX B ORAL EXAM STUDENT PERCEPTION SURVEY GENERAL PROFILE QUESTIONS Male _____ Female_______ GPA (approx)___________ Previous experience you have had with taking oral exams ____more than 6 ____between 3-5 ____between 1 and 2 ____none Major: Acc_____ Bus Adm._____ Other______ QUESTIONS PERTAINING ONLY TO THE ORAL EXAMS Using the following scale, circle the best response for each question: Strongly agree agree indifferent disagree Strongly disagree 1 2 3 4 5 1. Do you feel you studied more strenuously for the oral exam than you normally would have? 1 2 3 4 5 2. Did you have enough time to adequately answer the questions? 1 2 3 4 5 3. Rate your anxiety level before the exam (Very high = 1; almost none = 5) 1 2 3 4 5 4. Rate your anxiety level during the exam (Very high = 1; almost none = 5) 1 2 3 4 5 5. Were you adversely affected by the “observer” in the room? 1 2 3 4 5 6. Were the exam room conditions appropriate to facilitate the process of the exam? 1 2 3 4 5 7. Did the exam giver’s body language have an impact on the process of the exam? 1 2 3 4 5 8. Would you say you studied “more” for the oral exam? 1 2 3 4 5 9. Was the feedback you received at the end of the exam adequate? 1 2 3 4 5 10. Was the exam graded fairly? 1 2 3 4 5 11. Did the exam giver “rush” you too much (during the oral exam)? 1 2 3 4 5 12. (Given this was your first oral exam) did you know what to expect from the oral exam? 1 2 3 4 5 13. Did the exam giver take adequate notes on your performance during the exam? 1 2 3 4 5 14. Would you have preferred to have known beforehand how many questions would be asked during the exam? 1 2 3 4 5 15. Should the oral exam be an opportunity to demonstrate your knowledge/understanding of a topic of your choice during the exam? 1 2 3 4 5 16. Was the exam giver’s attempts at clarifying various aspects of some of the questions adequate? 1 2 3 4 5 Developments in Business Simulation and Experiential Learning, Volume 35, 2008 126 17. Were you given enough time to reflect on your responses and clarify them if necessary? 1 2 3 4 5 18. Was the exam giver’s body language a hindrance on the process of the exam? 1 2 3 4 5 19. Instructor’s position was: a. on the same side of the desk as me b. across the from desk from me c. we both sat at a table QUESTIONS COMPARING ORAL AND WRITTEN FORMATS (for this course) Heavily favors oral Probably favors oral No difference Probably favors written Heavily favors written 1 2 3 4 5 20. Oral exams are a better way to express my knowledge of the subject (than written exams) 1 2 3 4 5 21. Oral exams are a better way to express my understanding of the subject (than written exams) 1 2 3 4 5 22. Better way to express my analysis of the subject matter (than written exams) 1 2 3 4 5 23. Better way for me to show how to apply my knowledge and understanding of the subject (than written exams) 1 2 3 4 5 24. Oral exams are more “subjective” than written exams 1 2 3 4 5 25. Generally, oral exams are “harder” than written exams 1 2 3 4 5 26. Generally, oral exams require more study preparation than written exams. 1 2 3 4 5 27. Generally speaking, I had more anxiety before taking the oral exam than before taking the written 1 2 3 4 5 26. I was able to discuss the oral exams questions with other students who had taken the oral. ______more than 3 students ______1-2 students ______unable to discuss with anyone ______I was the first one taking the exam 28. Generally, do you think the oral format should be used in selected upper level courses? ____yes ____no ____not sure GENERAL QUESTIONS 29. Should one have a choice between all orals, some orals or no orals in selected courses? ____yes ____no ____not sure 30. Students should have to take at least one oral in one course during their undergraduate study ____yes ____no ____not sure Any comments about the oral exams Developments in Business Simulation and Experiential Learning, Volume 35, 2008 127 Table of Contents Volume 35, 2008 Linking Team Covenants To Peer Assessment Of Simulation And Experiential Performance Using the Balance Scorecard Approach: A Group Exercise IndoAmerican Enterprises Class Size and Game Design Implementation Of Effective Experiential Learning Environments Simulation Sensemaking: The BusinessWeek Approach To Effective Debriefing A Lesson in Hide-And-Go-Seek: A Team Building Game Thoughts On How To Motivate Students Experientially Experiential Learning Is Not Just Experiential Teaching: Measurement of Student Skill Acquisition via Assessment Centers ABSEL Redux: Reflections after a 25 Year Hiatus College Student's Expectations of Technology- Enhanced Classrooms: Comparing 1996 and 2006 The Business Student Satisfaction Inventory (BSSI): Development and Validation of a Global Measure of Student Satisfaction The "Big Picture Question" Project: Explorations in Teaching Creativity within a Force-Field Research Framework "Viva Voce": Oral Exams as a Teaching & Learning Experience Internships And Occupational Socialization: What Are Students Learning? Does Learning Occur in One-Shot, Non-Cooperative Games? Beliefs and Behavior, an Ancient Perspective and Modern Application Developing Enterprise Culture Among the Students Through Intercollegiate Competitions: A Case of Student Enterprise Competition (SEC) 2007 Student Views of Management Skills and Their Future Careers after Using Business Simulations Back to the Future: Gender Differences In Self-Ratings of Team Performance Criteria A Case for Experiential Learning: Using Central Europe as a Classroom Modeling Strategic Opportunities in Product-Mix Strategy: A Customer- Versus Product-Oriented Perspective Assessment in the Modern Large High-Tech Classroom Target Profit Pricing With the Web-Based Breakeven Analysis Package Are the Business Simulations We Play Too Complex? Shared Experience as Incentive for Horizontal Integration in Business Simulations Affinity Propagation: A Clustering Algorithm for Computer-Assisted Business Simulations and Experiential Exercises Marketing Simulation Results as Embedded Forms of Program Assessment Human and Agent Playing the "Beer Game" Issues in Porting a LAN-based Total Enterprise Simulation Game to a Web-based Environment Partners or Competitors? A B2B Simulation Evaluation Model of the Global Performance of a Management Simulation for the Academic Environment Applying Bloom's Revised Taxonomy In Business Games Do Price Strategies Work in Business Simulations? Early Japanese Gaming Simulation Efforts Using Par Players to Enhance Learning in Business Simulations Corporate Cartooning: The Art of Computerized Business Simulation Design Should Business Game Players Choose Their Teammates: A Study with Pedagogical Implications Goal Orientation and Simulation Performance Design and Demonstration of an Online Managerial Economics Game with Automated Coaching For Learning and Graded Exercises for Assessment Are Good Strategy Decisions Consistently Good? A Real-Time Investigation Active Learning 2.0 or Wiki is not a 4-letter Word Comparing Student Learning in Online and Classroom Formats of the Same Course How Do We Get To Tomorrow? The Path to Online Learning Similar Media Attributes Lead to Similar Learning Outcomes Facilitating Business Gaming Simulation Modeling