THE USE OF BOARDS OF DIRECTORS TO EVALUATE REPORTS AND PRESENTATIONS IN AN UNDERGRADUATE BUSINESS POLICY COURSE Developments In Business Simulation & Experiential Learning, Volume 24, 1997 THE USE OF BOARDS OF DIRECTORS TO EVALUATE REPORTS AND PRESENTATIONS IN AN UNDERGRADUATE BUSINESS POLICY COURSE William D. Biggs, Beaver College ABSTRACT This paper describes the author’s experience with using a variety of individuals to serve as members of a board of directors for undergraduate students playing a computerized general management simulation as part of a business policy course. The use of the boards to evaluate student reports and presentations is discussed. Some of the benefits and problems associated with using board members are presented. INTRODUCTION For about 15 years the author has used both cases and a computerized business simulation as part of a senior-level business policy course. The same simulation has been used for all fifteen years, however, the current version, Micromatic. A Management Simulation (Scott. Strickland, and Hofmeister, and Thompson. 1992) is designed for the microcomputer and is somewhat more complex than the original version, Tempomatic IV: A Management Simulation (Scott and Strickland. 1984), which was designed for the mainframe computer. The simulation is a moderately complex general management simulation in which students, working as members of a management team, must make and integrate decisions in the functional areas of marketing, finance, production, and human resources. In addition, they must consider external influences, such as economic, legal, and social conditions and the nature of the industry and competition within the industry when making their decisions. Decision sets represent a quarter of a year of business operations and during the semester we currently require 16 sets of decisions which means the students are operating the company for four years. As in the ‘real world’ there are random and lagged events involved. For example, there is a 7% probability that only 80% of the raw materials will arrive and raw materials ordered in one quarter do not arrive until the next quarter. When the students take over the company it has been in existence for two years so there is historical data available. To add to the “real worldness” of the environment the students must prepare written reports and make presentations to their board of directors and the public at large. There are three types of reports and presentations. First, at the end of the their initial year of operation (i.e., year 3) the students must prepare a business plan. The business plan is read by the board of directors and the students make a presentation to the board concerning the plan. Second, after years 4 and S the students prepare a management report that describes what they set out to accomplish, what happened, and what they will do in the future Again, these reports are read by the board members and a presentation is made to them. Finally, after year 6 the students prepare a poster which, with verbiage and graphs, gives an overview of how the company performed. The posters are put on display in the library gallery and a public meeting is held. Biggs (1992) provides more information concerning the use of poster presentations. THE BOARDS OF DIRECTORS The board members are drawn from five sources: (1) faculty members from business administration and economics from a number of colleges in the area: (2) faculty members from other departments at the college, (3) members of the college board of trustees, (4) business executives not associated with the college: and. (5) alumni of the business program who played the game when they were students. The board members are told that their role is that of a questioner not adviser. The board members are asked to evaluate the reports and 274 Developments In Business Simulation & Experiential Learning, Volume 24, 1997 presentations. About two days prior to meeting with the students the board members receive the reports and evaluation forms. During the meetings the students have 15 minutes to make a presentation and the board has 20 minutes to ask questions and discuss results and plans. Each board member serves on two boards so they have comparative information. BOARD AND INSTRUCTOR EVALUATIONS An interesting aspect of the evaluations of the reports is that over the years the author has found a highly positive relationship between his evaluation of the student reports and the board evaluations. Table 1 presents data for the first management report from one semester. The raw score in Table 1 for the boards are based upon an average of the scores submitted by each board member for a particular team using a form provided by the instructor. The form contains seven questions related to the report: (1) well written; (2) well conceived, (3) clearly organized; (4) complete, (5) apparently accurate; (6) helpful and/or convincing; and (7) generally satisfactory for its purpose. For each question the board members can respond that they strongly agree (10 or 9 points), agree (8 or 7), disagree (6 or 5), strongly disagree (4 or 3), neither agree nor disagree (2 or 1), and no basis for judgment (0). An average score is calculated for each board member and an overall average for all the board members for a team. The reports are then rank ordered using the average raw scores. The author's raw scores for each team where derived using a more detailed evaluation form which provides the students with points on 16 different criteria. The total points can range from 0 to 100. The instructor’s raw scores are not simply obtained by adding up the points for each item for each team, however. Instead a linear transformation is used based upon the authors past experience. Typically, a point count of 45 equals a percentage score of 60 and a point count of 95 equals 95 to 100. All other points are then distributed using the formula Y=aX + b. Again, the reports are rank ordered using these raw scores. As can be seen from Table 1 the rank ordering of the reports is quite similar. In fact a sign test of the null hypothesis that the rank order of the evaluations by the board members will not differ from the rank order of the instructor is supported (p>. 05). This high level of correspondence is remarkable when one considers that each board member only saw two reports whereas the instructor evaluated all 10. In addition, two different forms are being used for the evaluations. Evidently they provide a holistic approach to the evaluation so correspondence is achieved. Perhaps even more impressive is the high degree of correspondence between instructor and the board raw scores. Only for teams 10 and 3 are there any substantial raw score differences between the board scores and the instructor and in both cases the instructors scores are lower because the instructor had more information about some very specific types of requirements which were not fulfilled and saw all 10 reports whereas board members only saw two reports. Even so the regression for the raw scores yields an r2 of .656. The boards also evaluate the students’ presentations. One part of the evaluation deals with how 275 Developments In Business Simulation & Experiential Learning, Volume 24, 1997 well the total presentation was organized by the team and the extent to which all the team members participated in the presentation. A second part provides feedback to the students individually concerning his/her effectiveness. Each student receives only his/her own individual evaluation, however, the team members are encouraged to share their evaluations with one another. Since I also like symmetry the students also evaluate the board members at the end of the semester. BENEFITS AND PROBLEMS ASSOCIATED WITH THE BOARDS The use of the boards of directors has a number of benefits. First, the students benefit because of their interaction with the diverse audiences represented on the boards (i.e., faculty, alumni, business executives, and trustees). The students are forced to formulate their ideas and communicate them orally and in writing. The students also receive feedback from the board on the reasonableness of their strategies and tactics. As reported by Rosenthal and Werner (1992) we find that the students change their behavior from the first meeting to the second as they adapt to feedback from the board. Second involvement of faculty members from business and economics gives credibility to their subject areas when the students discover that they are being asked questions which relate to learning from other courses. Third, the involvement of faculty member from other disciplines demonstrates to students that there are dimensions of knowledge, logic, problem solving, critical thinking, etc. which can be drawn upon from other subject areas. The involvement of faculty from other disciplines also gives credibility to the business curriculum as the level of expectations is realized. Fourth, the members of the board of trustees have a chance to interact with the students to assess the “product” that the institution is ‘producing.’ Fifth, the executives report that they benefit because the students make them think. Sixth, the involvement of alumni is helpful. Students in one sense see themselves five or six years from now. From an institutional standpoint the inclusion of the alumni provides one addition way in which our graduates are tied back to the institution. Seventh, the involvement of faculty from other institutions provides them with direct knowledge concerning our graduates who may attend their institutions as graduate students. Finally, the instructor benefits because the students become aware that the concepts being used in class are used and understood by people in the ‘real world.” In addition, the instructor benefits from the board members’ evaluations of the student reports. As indicated above there is a high degree of correspondence between the evaluations and the students are typically quite surprised to see this. The instructor also benefits from the board member evaluations of the presentations both on a group and individual basis. It is not logistically possible for the instructor to arrange presentations so that all could be attended therefore the boards provide information, which would not otherwise be available. The use of the boards is not without its problems. First, some board members will fail to attend the meetings. For this reason I recommend the use of four or five members per board so if one or even two fail to attend there is still a feeling on the part of the students that they have a board with whom to speak. If five members are used, however, and they all attend the students may feel intimidated if there are only three students on the team. Second, the use of the boards creates timing problems. The decisions must be spaced in such a way that information can be provided to the board, the meeting held, and the next decision submitted but without overlaps. Third, even when board members are told not to give advise they will do so. It is always interesting when students tell me about advice they have received and I know it is wrong either because of the conditions I have created or worse yet due to theoretical considerations. CONCLUSION Even with the problems I encourage more simulation users to use boards of directors. The ‘real world” aspects that are brought to the simulation far outweigh the problems. The interested reader may want to contact the author or others 276 Developments In Business Simulation & Experiential Learning, Volume 24, 1997 (Rosenthal and Werner, 1992 or Wheatley, Roberts, and Einbecker. 1990) who have used boards of directors in the classroom. REFERENCES Biggs, W. D. (1992) The use of poster presenta- tions as a final project in the business policy course. Developments in Business Simulation & Experiential Exercises, J. Gosenpud arid S. Gold (eds.), 17-19 Rosenthal, W. and Werner, W. J. (1992) The use of “a board of directors” to evaluate and validate decisions in a competitive graduate management simulation course. Developments in Business Simulation & Experiential Exercises, J Gosenpud and S. Gold (eds.), 221. Scott, C. R., Jr. and Strickland, A. J., 1111 (1984). Tempomatic 1V A Management Simulation. Boston, MA: Houghton Muffin Company. Scott 1.W., Strickland, A. J. III, Hofmeister, D L., and Thompson. M. D. (1992). Micromatic: A Management Simulation Boston, MA: Houghton Mifflin Company. Siegel, S. and Castellan, NJ, Jr. (1988) Nonparametric Statistics for the Behavioral Sciences. New York NY: McGraw Hill Book Company. Wheatley, W. J., Roberts, R. M. and Einbecker, R C. (1990). A complex simulation and community involvement yield an award- winning capstone experience. Simulation & Gaming 277 Table of Contents Volume 24, 1997 Incorporating Computer Telephony into the MIS Course A Learner Oriented Infrastructure for videoconferencing Based Distance Education Courses The Identification of Temporally Related Structural Elements of the Experiential Component of Electronic Spreadsheet Tasks Does Involvement Influence Learning from Simulation Participation? Some Relationships with Helpfulness and Performance Outcomes Musings on Business Garne Performance Evaluation Performance on a TE Simulation: What does it represent? The Business Policy Game: An International Simulation - An Assessment Tool Students in Free Enterprise as Experiential Learning The Role of Computer Models in Wargames: A Practitioner's view Using Focus Groups as a Tool to Research Learning: A Demonstration and a Discussion Using Medical Simulations to Teach Multi-Cultural Diversity Using the World Game to Internationalize International Exchange Game Community Services Needs Assessment: An Innovative Approach Introducing Students to Potential Total Quality Management Ethical Dilemmas Privacy in the Workplace: A Situational Analysis Antecedents of Learning in Simulations Demonstrating the Learning Effectiveness of Simulation: Where we are and Where we need to go The Use of Computer Simulations as a Pedagogical Aid in Teaching Management Information Systems Evaluating Simulation Learning in a Distant Learning Instructional Model Financial Engineering of Global Investments The Labor History Game: Playing with the Past Perks Participation Assessing Negotiator's Proficiency with a Negotiation role-play The Art of Negotiating Enhancing Learning and Employee Development through the Assessment of Learning Pedagogy Preferences across selected Dimensions of Culture: A Preliminary Investigation Cooperative Learning: What are we Learning? Effective Use of Mastery Based Experiential learning in a Project Course to improve skills in system analysis and Design Simulations and Learning: Can we prove a relationship? (Seminar) Predicting and Reviewing NYSE Stock Prices by use of Basic Statistical Analysis and Logic Toy Car Depreciation Exercise ABSEL as Home Community: An Interactive Exploration Service Learning: Linking Academic Study to Community Development and Business Enhancement Ability of Efficient Evaluation of Knowledge-Based Management Strategies Corporate Ethics Training Programs Modeling Attributes in Demand Functions of Computerized Business Simulations: An Extension of Teach's Gravity Flow Algorithm Expert Systems Combined with Neutral Networks: Tools to Benefit the Marketing Researcher Computer Game Design: New Directions for Intercultural Simulation Game Designers Consistency in Simulation Performance over Time and Across Simulation Games The Impact of an Artificial Market Leader on Simulation Competitor's Strategies Business Plans, Case Studies, and Total Enterprise Simulations: A Natural Co-existence An Exploration into the Non-Use of Business Simulations The Market Game: Interactive Learning Through Market Simulation Plotting Brand Trajectories with the COMPLETE PPM package: A Market Segmentation Analysis and Positioning Tool Me and Mine Inc: An Exercise in Management Theory Learning to Differentiate Leadership from Managerial Position Business Ethics Survey: A Perspective from the Retail Industry Experiential Learning in Demand Analysis for an Agricultural Commodity Marketing on the Internet: A Pedagogical Exercise College Students Need Simple Computer Simulations, Especially for International Business Courses An Analysis of Student Attitudes, Performance, and Strategies in a Simulation Competition Based on a Controlled Product-Market-Entry Game Structure The Crystal Enterprise: Application of a Non-Computerized Simulation Model in a Process of Organizational Change Threshold: A Windows-Based Behaviorally Oriented Total Enterprise Simulation Coaching Business Game Teams Using a Decision Variable Optimizer Rock and Roll is here to Stay: Enhancing Experiential Pedagogy with Musical Exercises An Example of Business Process Analysis Simulation for Customer Software Support The Use of Business Gaming in Hong Kong Academic Institutions Using the Integrative International Simulation INTOPIA Mark 2000 in a Concentrated MBA Curriculum A Group Experience as an Integrated Part of the Core Management Course Empowered Learning in the Classroom The Inter-Group Interaction: An Innovative Approach to Cooperative Learning Courses that Utilize Student Team: An Approach to enhancing their Effectiveness Thoughts about the Measurement of Learning: The Case for Guided Learning and Associated Measurement Issues Measuring Student Learning Using Business Simulations: A Theory Based Perspective On the Use of PC Fingame in an Undergraduate Finance Course How Managers get Things Done: A Virtual Soundbite Internet Experiential learning in the Principals of Marketing Classroom: A Pedagogical Approach Current Student Perceptions Relative to Business Simulations Beyond Capitalism: Designing Business Policy and Social Justice The Use of Boards of Directors to Evaluate Reports and Presentations in an Undergraduate Business Policy Course Beacon Lumber: An Experiential Introduction to Financial Accounting The Application of Organizational Motivation Principles: The Experiential Business Simulation Motus Manufacturing Avoiding a Bogey: Grading Case Discussions Scientifically Communicating Consumer Behavior II: A Modified Exercise Using Personal Consumption Journals in Condensed Courses Designing Instruments for Assessing the Effectiveness of Simulations, (Seminar) 360º Performance Feedback: Appraisal vs. Assessment The Cafeteria Approach to Managing an Academic Career: Remaining Non-Perishable while Doing Your Own Thing An Experiential Exercise Related to Person-Organization Fit and it's Consequences For Today's Dynamic and Changing World, Business Simulations need to be expanded to encompass Much Greater Complexity: A Demonstration The Incident Process: A Case in Reverse Contextually-Anchored Business Simulations The Energy Factor: Building Motivation in the Simulation Gaming Environment The Design of an Internet Game