DEVELOPMENT OF MANAGEMENT SKILL ASSESSMENT Developments in Business Simulation and Experiential Learning, Volume 25, 1998 DEVELOPMENT OF MANAGEMENT SKILL ASSESSMENT Edward M. Pogossian. Academy of Sciences and American University of Armenia ABSTRACT The problem of management skill assessment by games is analyzed in the frame of a suggested model to explain some recent ‘strange’ experimental results. MANAGEMENT SKILL ASSESSMENT BY GAMES 1. In the problem of Management Skill Assessment by Games (MSAG) given a business game G and some competitors C1, C2, Cm competing in G it is required to select a criterion K and competition assessing methodology M such that ordering of the C1, C2,...,Cm in accordance with (K,M) would be isomorphic to the ordering with respect to the “Management Skill Assessment Standard Methodology’. 2. To advance in solving the MSAG problem it is necessary to interpret and explain some concepts. First, what does “Management Skill Assessment Standard Methodology “ mean? In general, we do not know the answer. The current state of that important question and ways for its investigation are outlined in Anderson, Lawton-97]. Until its complete resolution we, like many others, are replacing that methodology by common recommendations of MBA colleges. For our purpose it is enough to suppose that there exists such standard methodology to interpret its syntax in our model. Second, a competitor C competes in accordance with one of available control programs which determine possible ways of its behavior. We suppose that during the competition each competitor selects one of the programs and keeps it unchanged. For that reason a competitor in each competition is identified by its current program. Third, competitors perform allowed actions, or make moves, from corresponding sets Al, A2,..., Am simultaneously step by step in T periods. We refer to a vector of such actions as a bundle of actions. Bundles of actions transform the initial situation S into sequences of new situations. A tree of all sequential bundles of actions of competitors from initial situation S in T periods we name S-game tree, or S-tree. In fact S-tree is the sum of performance trees of the competitors' programs starting from S. The whole performance of the programs may be described by the forest of such trees from different initial situations. To avoid technical complications in definitions later on we suppose to have only one initial situation. We name the performance of a competitor C i.e. representation of its program , in S-tree as a complete S-strategy of C. As a graph a complete S-strategy of C is a subtree of the S-tree where the actions of C are indicated for all possible situations emerged from S. Any subtree of a complete S-strategy of C with root situation P we name P-strategy of C. We also use terms a strategy of C, a strategy, etc. Forth, at the MSAG problem level of approximation we perceive each manager as a unchanged program representing him during the competition. As a consequence of that supposition we must accept that the management skill assessment problem is reduced to the evaluation of the performance of the corresponding program. Because the performance tree of a competitive program C in a S-tree is a S-strategy the Skill 29 Developments in Business Simulation and Experiential Learning, Volume 25, 1998 Assessment Standard Methodology may be interpreted as some criterion that is tested on a given S-strategy to evaluate its quality. Let us denote that criterion as SASM and suggest that its value is determined by terminal nodes of S- strategies. For example, it may be the average profit gained by S-strategy and calculated by averaging the value of the profit for all of its terminal nodes. Thus, if we measure SASM for each S-strategy of the Cl, C2,..., Cm and order them correspondingly we will get ‘the ideal “, or ‘desirable “, ordering of managers in accordance with their skills, or SASM-ordering. 3.How could we get the ideal SASM-ordering of competitors using SASM criterion and business games tournament approach of assessment? Given criterion K and methodology M we play a series of games from Initial situations to order competitors CLC2,. . .,Cm in accordance with their performances. Since any game from initial situation S corresponds to appropriate terminal nodes in Strategies of Cl,C2.. . .,Cm. the series of games will be represented by corresponding sets of terminal nodes of these strategies. Thus, in order to obtain SASM-ordering all nodes of all strategies must be estimated. It is possible as a result of robin-round tournament where each competitor must meet with all possible (m-1) bundles of all possible strategies of (m-1) competitors. The results of such tournament may be represented by (m, N) matrix where N is equal to the number of all (m-1) combinations of strategies of the sets of possible strategies of competitors. It is evident that even for moderate values of the above parameters that estimation is a computationally difficult problem. 4. Let us see how we actually order managers (players) in games like MARKSTRAT2. When we try to order C 1 ,C2 Cm in accordance with series of their game performances we actually try to approximate the ideal ordering of the strategies based on the above complete analysis of their terminal nodes by some partial analysis. Even in the case of using SASM criterion we may obtain different orderings of the strategies due to the insufficient amount of analyzed terminal nodes. Thus, it becomes evident that we have to substantiate the acceptance of our approximation and estimate the level of its correctness. One of the reasons why experiments in [Anderson, Lawton, ABSEL-97] did not reveal any positive correlation between standard management skill assessment criteria and the ones used in business simulation may be the absence of the estimation of the necessary set of games of participants to obtain acceptable approximation. 5. Since SASM-ordering of competitors based on their game performances. or tournaments, is a computationally hard problem, the question of appropriate constrains becomes actual to reduce the complexity. For a symmetric problem of the evaluation of the effectiveness of management strategies in [Pogossian, ABSEL- 97] the criterion of knowledge based “essentially improvable” strategies was formulated allowing to advance in the efficient evaluation of strategies. The criterion may be applied to games like Value War [Chussil, Reibstein, 94] where strategy search simulation is used to predict results of oligopoly competition for consultancy purposes. Because that criterion seems natural and acceptable for managers, we study its application for management skill assessment purpose. Reference available upon request: epogosasi@aua.am 30 Table of Contents Volume 25, 1998 Marketing Goes to the Movies Bringing Experiential Learning to a Principles of Marketing Course Investment Analysis Application Using In-house Spreadsheet Models SugarCoated Statistics: An Exercise for the First Day of Class Improving Undergraduate Student Involvement in Management Science and Business Writing Courses Using the Seven Principles in Action Establishment and Funding for Interuniversity / Multidisciplinary Student experiences The Prospects of Creative Teaching: A Discussion with Patricia Sanders The Simulation and Classroom Assessment Techniques Developments of Management Skill Assessment Games as Instruments of Assessment: A Framework for Evaluation The Role of Artificial Intelligence in Business Curricula Threshold Solo Competitor: A Management Simulation (V1.0) a Windows-Based. Play Alone, Total Enterprise Simulation and Assessment Instrument Toward An Understanding of One's self-concept Total Enterprise Simulations and the Internet: Improving Student Perceptions and Simplifying Administrative Workloads The Expatriate an Assignment Orientation Game An Expatriate's Nightmare: An Experiential Exercise in Coping with Overseas Assignments Analyzing Experiential Exercise: Using the Scientific Method for Problem Solving The Second Component to Experiential Learning: A Look Back at How ABSEL has handled the Conceptual and Operational Definitions of Learning Predictive Models of Learning: Participant Satisfaction of Experiential Exercises in Business Education Accelerating Moral Development through Use of Experiential Ethical Dilemmas Ethical Dilemmas to use with Business Simulations to Teach Ethics The Class Approach in Behavioral Simulation in a Business Policy/Strategic Management course: A Progression toward Greater Realism An Exploration of the Emergence of Process Prototypes in a Management Course Utilizing a Total Enterprise Simulation How Organizations Are Improving Their Performance Utilizing Electronic Commerce: Examples From The Internet The Market Access Planning System (Maps): A Computer-Based Decision Support System For Facilitating Experiential Learning In International Business An Excel Workbook For Student Planning And Interface With A Simulation Game Design Of Multi-Media Based Pedagogy For Leadership Training Panel Discussion On Using The Internet For Courses Valuing And Enhancing Teaching: Sharing Tips Via The Web The Buddy Project: A Semester Long Project Aimed At Developing An Appreciation For Diversity Enhancing The Excitement And Learning Retention In The Classroom: The Power Of Magic The Supervised Management Internship: A Job Or Learning Experience Team Ware™ An Online Moderated Class Discussion Facility And Beyond A Neophyte Distance Educator's Experience Learning Management By Practicing Management: A Report Of Significant Student Service In 1997 Integration Of Academic And Service Learning: Students' Perceptions About Its Effects And Outcomes The Value Of Incorporating A Service Learning Component Into Course Content: A Presentation And Roundtable Discussion Business Games Teach: Thoughts on the Sources of Conflicting Conclusions on their Effectiveness Antecedents Of Learning In The Simulation: A Replication Using Student Journals To Enhance Learning From Simulations Technological Change And Intertemporal Movements In Consumer Preferences In The Design Of Computerized Business Simulations With Market Segmentation Integrating The Marketing Curriculum Using Collaborative Learning Teaching Time Management In A Sales Program: The Application Of A Computer Simulation Game Adapting Interactive Computer Simulations For Content Based Esl Instruction Multimedia And Student Expectations Synthesizing Data For Media Simulations Composing A Team Health Promoting Behaviors-A Decision Making Exercise Does it really Work? An Application of the Group Interaction Framework Administering the MIT Beer Game: Lessons Learned A Paperless Economy? Instructing Students on the Aspects of Successful Electronic Commerce Maximizing Learning Gains in Simulations: Lessons from the Training Literature Observing General Ability in a Total Enterprise Gaming Simulation FReach Teach: A Computer-Based System for Teaching Advertising Media Planning An Integrated Business Instruction System An Experiential Exercise you can Tinker With Experiential Exercises or Computer Simulations? Cash Flow Statements: Are They Important in Business Simulations? Holistic Cognitive Strategy in a Computer-Based Marketing Simulation Game: An Investigation of Attitudes Towards the Decision-Making Process Barnga: A Game on Cultural Clashes The Many Faces of Culture: Understanding Country and Corporate Culture Students' View of the Use of Business Gaming in Hong Kong Assessing General Management Interest What is the Future of Business Gaming? Starting a Small Music Trivia Business Exercise and Other Innovative Icebreakers An Integrated Approach to Behavioral Skill Development Career Focus: A Student and Business Learning Experience The Use of Concept Mapping in Teaching Strategic Management An Experiential Approach to Developing Mission Statements Business Games in Brazil-Learning or Satisfaction A Simulation within a Simulation: Job Layoff's and Emotional Reactions