INCREASING EFFICIENY OF MANAGEMENT SKILL ASSESSMENT Developments in Business Simulation and Experiential Exercises, Volume 26, 1999 INCREASING EFFICIENCY OF MANAGEMENT SKILL ASSESSMENT Edward M. Pogossian, Academy of Sciences and American University of Armenia ABSTRACT A system of assumptions was introduced in Po- gossian (ABSEL98) to shape an adequate model of the management skill assessment by simulation games. Under that assumptions and the quasi- transitivity constraint a theorem is formulated to find the solution of the problem with available computational resources and to overcome the ob- stacle of a complexity of that measuring process. EFFICIENT ASSESSMENT FOR QUASITRANSITIVE TOURNAMENTS 1. We refine management skill assessment prob- lems as the following. Managers are competing in oligopoly markets for the same objectives, or criteria K , e.g. profit, market share, Return On Investment (ROI), etc. Integrative performance of managers in possible competitions is evaluating by a methodology M. We suppose that on-the-job, performance of man- agers, Anderson, Cannon, Malik. and Thavikul- wat (98), is inducing a linear, or “ideal”, order- ing O*(K,M) of all managers. At the Management Skill Assessment (MSA) problem given managers C1,C2,...,Cm, criteria K and methodology M of an assessment of a compe- tition it is required to find an ordering O (K,M) of the C1, C2,…,Cm that would be isomorphi- cally imbedded into the "ideal" ordering O*(K, M). Since the ideal ordering of strategies based on their game performances or tournaments is a hard problem, Pogossian ( ABSEL98), the question of appropriate constraints becomes vital for reducing the complexity. To introduce constraints we need to standardize the sets of tournaments for each competitor. In fact, each competitor C competes against samples of all possible strategies of other competitors, i.e. against all strategies, except its own ones. To regularize analysis we suppose that C competes against its copy, or the twin too. Assumption. Each competitor competes against all strategies including its own. Thus, to get the ordering O(K,M) we may con- struct the Matrix of Grades (MG), Pogossian (ABSEL98), where each row corresponds a com- petitor and each value in the row is determined by the results of the game of the corresponding competitor against one of possible samples of competitors in oligopoly competition from all possible initial situations. 2. To reduce the search we use the Transitivity Constraint (TC) to the ordering O*(K,M) which is formulating as the following Assumption TC. Given programs Pi , Pj and sets of samples of strategies Bi and Bj losing to Pi and Pj , correspondingly, we have a tautology Pi is stronger than Pj with respect to the ordering O*(K,M) if and only if Bi includes Bj TC is based on the assumption that all competi- tors are essentially different in their skills. Thus, to order Pi and Pj in frame of the Transitiv- ity Constraint it is enough to find a sample of competitors that loses to Pi and wins Pj. 3. Let us analyze a more general than the Transi- tivity Constraint case where competitors may be close each to others by their skill level or even be- long to the same class of equivalence in the O*(K,M). Given competitor i is stronger than competitor j in the O*(K,M) denote by B(i,j) the set of samples such that i competitor loses to and j wins each sample of the B(i,j). Other words, B(i,j) is the set of samples which in the games with the programs i and j attain results not consistent, or opposite, 80 Developments in Business Simulation and Experiential Exercises, Volume 26, 1999 to the fact that i is stronger than j in the ordering. Following the Elo (78), we may accept that for fixed i and varying j the variance function #B(i.j) - the number of elements in B(i,j) , will have a form of normal distribution with respect to the distance between positions of i and j in the O*(K,M). If for a given set of competitors the variance B(i,j) is not zero then we can not apply the Transi- tivity Constraint deterministically but only with some probability. The relevance of the Transitiv- ity Constraint increases if the distance between compared competitors i and j is increasing. As a first step in simulation of the competition we approximate the normal distribution by rectangu- lar curve and formulate the following weak ver- sion of the Transitivity Constraint - the Quasi- Transitivity Constraint (QTCw). Assumption QTCw Given programs Pi and Pj , samples of strategies Bi and Bj loosing to Pi and Pj , correspondingly, and the variance function #B(i,j) it is possible to indicate a constants a and b (small enough compared to the number of all strategies in the ordering O*(K,M)) such that: • if j belongs to the segment [i+a ,i-a] then #B(i,j) may exceed zero for no more than b points • if j does not belong to the segment [i+a ,i-a] then #B(i,j) is equal to zero and Pi is stronger than Pj with respect to the ordering O*(K,M) if and only if Bi includes Bj. QTCw appears as an analog to the “essentially improved” strategies criterion in Pogossian (AB- SEL97). The following theorem is true. Theorem. Given a class F of competitors satisfy- ing to the Quasi-Transitivity Constraint in its weak form and strategies f and g from F , we say that f is stronger than g (i.e. the location of f is better than g in the ideal ordering O*(K,M)) if we find b samples of competitors' strategies such that f wins and g loses games against each of them. The proof of the theorem is similar to the one in Pogossian (83) for two-person games with perfect information. Thus, given the competitor f, the question of its strength relative to the competitor g is reduced to the construction of a special tournament and an estimation of the parameter b . Even without the estimate of b it is true that with increasing the number of testing samples the probability of f to be stronger than g increases too. The theorem also supplies the necessary basis for an experimental investigation and a complete justification of the assumption . 4. Is the Quasi- Transitivity Constraint reasonable for managers? The Quasi-Transitivity Constraint appears naturally if considered for knowledge-based human activities, i.e. depending on the amount of knowledge or skill avail- able for the decision making. For managers in general as well as for marketing man- agers discussed in the Pogossian (97), we may support the above statement by following arguments. First, management strategies are essentially knowl- edge-based. It is evident when we analyse the method- ology of the construction of business or marketing plans: experienced manager differs from a novice first of all by the amount of knowledge he uses for planning. Second, the skill of a manager is determined not only by the amount of the acquired homogeneous knowledge but, primarily, by the hierarchy of abstractions such that each new level is possible on the base of categories from the previous levels. This property strongly corre- lates with the ability of managers to improve their skills. And at last, in the real management practice there are evaluations and classifications of the managers' skill which are directly reflected by the system of their re- wards. References available from: epogossi@aua.am 81 Table of Contents Volume 26, 1999 ABSEL's Historical Research Interests Back From the Future: An ABSEL Merlin Exercise for the Year 2005 The Contributions of ABSEL During the 1980's ABSEL's Contributions to Experiential Exercises in the 90's ABSEL's Contributions to Experiential learning/Experiential Exercises: The Decade of the 1970's Business Simulations - Algorithms and Model Enhancements: A 25 year Review A Study of the ETS General Field Test as an AACSB Assessment Tool and the Impact of Experiential Exercises and Simulation on Learning A Framework for Assessing the Competencies Reflected in Simulation Performance Developing A Learning Culture: Assessing Changes in Student Performance and Perception Financial Simulation Using Distributed Computing Technology Analyzing Managers' Judgements and Decisions with an Educational Business Simulation Understanding Your Business through Home-Made Simulator Development An Examination of a Reanalysis of the Impact of a Market Leader on Simulation Competitors' Strategies Applying Shocks to TE Simulations: A Demonstration Increasing Efficiency of Management Skill Assessment A Testbank for Measuring Total Enterprise Simulation Learning Developing Leadership Skills - Video Live! LEADSIMM: Collaborative Leadership Development for the Knowledge Society A Team Approach to Producing Multi-Media Laptop and Video Formatted Presentation Tinkertoys Revisited: Exploring Trust Based Relationships Overall Dominance in Total Enterprise Simulation Performance Success or Bankruptcy: The Relationship between Personal and Goal Orientation and Simulation Performance Assessing the Effects of Feedback: Muti-method and Muti-directions in Multi-pedagogical Courses The Missing Ingredients in Experiential Learning Purpose and Learning Benefits of Business Simulations: A Design and Development Perspective Building Capabilities for Change through Laboratory Simulations Modeling Innovation as a Process of Design in Educational Business Simulation The Need to Measurer variance in Experiential Learning and a New Statistic to do so Assessing Effectiveness of an Experiential Oriented Course Over Time Developing Participant Satisfaction Models of Experiential Exercises in Business Education Perceptions of Learning in TE Simulations Students' View on the Use of Business Gaming in Hong Kong Management Gaming's Lost Opportunity: Meditations about the Russian Experience and Prospects for the Future Unanticipated Enhancements in the Business Strategy and Policy Game when Running in Windows 95 Using a Business Game to Demonstrate Broad Business Concepts A New Model for Business Courses (Getting the Student Connected) Seeing the Forest and the Trees: Integrating Knowledge Using Large Scale Simulations in Capstone Business Strategy Classes Is It Here To Stay? A Roundtable Discussion of the Inter-Group Interaction Interactive Tools Used in Applying Financial Concepts Strengthening Essential Skills through a Finance Exercise: Calculation of Beta A Model of Currency Exchange Rates Understanding Currency Exchange Rates: A three-part Exercise Student Experiences in the World Intercollegiate Business Game Competition Student Experiences in the International-Collegiate Business Policy Game Competition Using Boards of Directors in Simulation Environments: Comments From Board Members Sharing Best Practices: Teaching Smarter, Not Harder Star Power: A Simulation for Understanding Power and Empowerment The Marketing Game: A Marketing Principles Simulation Alexander Islands: GSSM Tiny Business Simulator on the WWW Putting Strategy into Strategic Business Games Industry Analysis, Porter's Five Forces Model and Strategic Group Maps in the Business Strategy Game Simulation The Use of Concept Mapping to Improve Student Performance and Understanding of Strategic Management Concepts: A Comparison of Techniques Transformational thinking in the Organizational Behavior Course: The Use of Metaphor as an Assessment Tool Creation of a Virtual Learning Community for the Global Virtual Enterprise Project Inter Institutional use of Educational Resources: Joint Use of Management Simulation Games Business Insights: Theory and Practice with the Aid of a Business Simulation Progress: An Experiential Exercise in Development Marketing The ABC's of teaching the Theory of Constraints to Undergraduate Business Students Demonstrating Principles of Organizational Purchasing Behavior through an Experiential Exercise The Virtual Manager: A Different Simulation for Managing Complexity When the Rules are changing and Chaos Breeds Innovation: Recapturing the Value of Constructive Thinking and Play in Simulation Training The Pitfalls, and Potential, of Actual Events as Problem Drivers Using Business Games to teach Environmental Awareness and Green Management: The International Experience with the ENSIM Game A Review of my ABSEL-Related Work Simulation of Government: A Workshop Using GEO Creating Internet-Based Games Using Perl and JavaScript Who's on First? Exploring the Concepts of Problem-Based Learning, Experiential Learning, and Lifelong Learning Students Learn Customer Service and Selling while Conducting Research So You Want to Run an NFL Football Team–An Honors Interdisciplinary Project Supervised Internship: The Employer's Perspective Using Computer Assisted Simulation to Teach International Business Strategy: A Case of the Multinational Management Game (MMG) Multi-Cultural Experiential Learning: A Computer Simulation in China An Appreciative Stance on Diversity as We Move into the 21st Century: A Timeline Exercise to Identify Key Experiences in Good Work Relationships between Black and White Peers A Day in the Life of an Interactive, Real Time, and Internet Delivered Course: A Demonstration Comparing Internet Search Engines: An Experiential Learning Exercise Total Enterprise Simulations and the Internet: Assessing Student Perceptions and Preferences Teaching Accounting Information Systems in a Practicum Format Providing Experiential Learning in Accounting through a Field Study Payroll Project A Spreadsheet Based Business Simulation Game Computer-Behavioral Simulations Training for Project Managers Simulation Scenarios - Rationale and Illustration