The Use of Concept Mapping To Improve Student Performance and Understanding of Strategic Management Concepts: A Comparison ... Developments in Business Simulation and Experiential Learning, Volume 26, 1999 THE USE OF CONCEPT MAPPING TO IMPROVE STUDENT PERFORMANCE AND UNDERSTANDING OF STRATEGIC MANAGEMENT CONCEPTS: A COMPARISON OF TECHNIQUES Douglas L. Micklich, Illinois State University at Normal ABSTRACT Previous research (Micklich, 1998) has shown that by using concept mapping, a greater level of understanding of the relationships between strategic management concepts can be gained. Although an increase in exam scores and a certain level of cohesion in a class strategic planning project was attained, overall perceived satisfaction with the exercise was low. This occurred under conditions where greater control was relinquished to the student for map construction and interpretation as the exercise was continued. The question was then posed; if the instructor took a more active role would higher levels of satisfaction, hence translated by better scores, and a smaller standard deviation of these scores, be realized? Results showed that the level of satisfaction rose for the experimental group as a whole, the mean score did increase, but at a smaller rate and the standard deviation of the scores did increase and at a greater rate relative to the control group, resulting in better understanding of those concepts and higher levels of perceived satisfaction. INTRODUCTION Trochim (1989a) stated; "In general, we have found that concept maps are better understood by people who have participated in all phases of the process than by those who have taken part in one or two steps." Participation can be implied, in the context of this statement as, "maintaining an active interest in map construction, interpretation and use on the part of the student." It can also be implied from this statement that the main concern is for the participants, those constructing the map. There are two general classifications of players in the exercise, the student and the facilitator; a question presenting itself concerns the degree of participation on the part of the facilitator. The facilitator's role in concept mapping is only to manage the process; the content, interpretation, and utilization of the concept map are determined by the group (Trochim, 1989a) or the individual. Several issues come to mind, the first of which is the passive vs. active management of the exercise on the part of the facilitator. If a more active role is undertaken, a more active role will likely be or expected to be taken by the student or group. A second issue is that one would expect that better control over the management of the mapping exercise results in better student performance (resulting from the technique being both used and understood). Thirdly, can the behavior of the facilitator (passive vs. active) be related to the degree of participation and level of satisfaction on the part of the student? The objective of this research is to determine or show, that when a greater degree of control (i.e. management of the process) is undertaken, performance and learning is enhanced, and therefore, the mapping technique will become an even better tool for learning. The expected results of increased control over the mapping exercise will be shown through improved performance on exams. In addition, a greater degree of satisfaction and usefulness of the exercise will be realized. Control, as defined for the context of this research, is "helping determine pathways for organizing meanings and for negotiating meaning with students, as well as to point out student's misconceptions" (Novak, & Gowin, 1984). Student performance being defined as: 238 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 better test scores, greater benefit seen from using the techniques, and as a result, smaller standard deviations in the test scores. Previous research by Micklich (1998) had the following objectives in mind. The first was to be able to draw/construct a visual image of the many relationships, both simple and complex, that comprise strategy development in something other that a linear relationship. Secondly, be able to understand, as a result of this image, the relationships and the dynamics of the variables that comprise the strategic planning process, the dynamics being the multidimensionality of the process when constructing a cohesive strategic plan for a particular corporation. When considering the added dimension of exercising greater control, two additional research objectives are presented. The first is, will exercising greater control over the mapping experience result in higher levels of satisfaction gained by using the maps as developed? The second being; as a result of increased satisfaction with the mapping experience and the map, will the mean test scores increase, the standard deviation of the scores decrease, or both? Expected results of increased control and the mapping experience will be shown through increased satisfaction, and improved performance on exams and the class project, than if the controls were not in place. Satisfaction in this situation is defined as: whether the mapping experience and the resulting map assisted in enhancing the understanding of the concepts' relationships. Control is defined as: the amount of information and assistance given by the instructor toward the group in illustrating the maps and relationships that can be drawn from the maps, and the level of autonomy the student is given for completing the map outside the class environment. Therefore, by deriving greater levels of satisfaction and exercising greater control, noticeable differences should exist in exam scores and standard deviations. Moreover, the standard deviation of these scores will be smaller than if this attention was not given. METHODOLOGY The students participating in this exercise were from the capstone business course, Problems in Business. Students from the Spring Term 1998 served as the control group and students from the Fall Term 1998 served as the experimental group. The Set-up Process The set-up process for the study group (Spring 1998) is similar to that used in previous exercises (Micklich, 1998). Several modifications to this process were made. First, a statement describing a concept map is shown on an overhead viewer; second, a list of words and phrases is placed both on an overhead and distributed to the class; and, third rules governing the construction of the maps are also distributed to the students, instead of being only stated. The mapping exercise was conducted in three parts, each with a greater amount of explanation as to the mapping concept, map construction and interpretation, and future directions to be taken via the map. The first part was conducted prior to the first exam. An introduction to concept mapping was given and a word list was given to each student containing topics that were covered for the first exam. Maps were to be constructed outside of class and turned in with the exam. When the exams were returned to the students, a drawing of the instructor's map was included. In addition, a copy was shown on an overhead and the relationships explained. At this point "take-off" points or "points of departure" were discussed of where the topics/course would be next headed. These "take-off points" are levels 239 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 of magnification useful in helping learners acquire and recalling a rich store of detailed expressions, coordinated ideas and meanings, vivid feelings and images (Novak, Gowin, 1984). The second part of the exercise was conducted similar to the first where a list of topics/concepts covered on the first exam along with words/phrases covered on the first exam were distributed. The maps drawn for this part illustrated only those words and phrases introduced since the first exam. This again, was done prior to the second exam, and the same post-exam procedures were followed. The third part of the exercise was conducted in a slightly different manner, where in prior rounds a word list was distributed to the students. In this instance a word list was distributed along with a "fill-in-the-blanks" template of the map. Spaces for the words and phrases were provided and the relationship arrows and phrases were given. It was the student's responsibility to complete the map with the proper words constructing logical relationships. This can be considered to be a form of "pre-mapping" which is used to clue students as to what misinterpretations they should watch out for as they read (Novak, Gowin, 1989) or as they construct their map. Students were required to hand in the map with the final exam, as with all exams, and for those that requested it, a copy of the final map was given to them when they turned in the final exam. Concept Reinforcing Mechanisms Throughout the semester, topics and concepts that went into the construction of the mapping exercise were presented through class discussion and the use of overheads. Additionally, copies of overheads were made available to the students. These topics are major areas of focus in the course and serve as points of departure to other topics or areas for discussion. The information presented was either a topic in and of itself or as part of another topic or concept, the latter being reinforced as "take-off points" where the map's relationships were explained. Students were asked to recall and refer to illustrations at various times throughout the course as overheads from prior discussions were re-introduced in an attempt to further reinforce the connectivity within a given map and between the various maps. The word lists also served as a review of major areas already discussed in the course. RESULTS The same survey instrument was given to students in both the Spring Term 1998, and in the Spring Term 1997. Table 1 shows the questionnaire and results from these two survey groups. A paired samples T-test was conducted to determine the significance and additional effect added control, displayed in the mapping exercise, had on both the mean test score and the standard deviation of the score. The Spring Term 1997 served as the control group and Spring Term 1998 served as the experimental group. The results of these tests can be found in Table 2 (Paired Differences), Table 3 (Paired Sample Statistics - Spring 1997), and Table 4 (Paired Sample Statistics - Spring 1998). Survey Results The main objectives of the survey were: to determine the perceived effectiveness or usefulness of the exercise in their understanding of the relationships, their aid in preparing them for class discussion, and their aid in exam preparation. The response rate to the questionnaire increased from 72% in 1997 to 100% in 1998. Comparison of these results reveals that although the percentage of those finding the usefulness of drawing the maps to help in understanding the relationships did not change, 240 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 (53% in 1997, 52% in 1998), dramatic differences did occur in all other areas. A slightly lesser number (51% in 1998 as compared to 55% in 1997) stated that their maps did not change appreciably, as the technique progressed. This could indicate a greater initial level of understanding was gained from the outset. Additionally, it can be seen that referring to the maps did help ones understanding as the percentage of those stating so increased (57% in 1998 as compared to 24% in 1997). Similar results can be seen in both assisting in overall class preparation as well as for exam preparation. Paired Differences Comparison: Exam 1 to Exam 2 From Table 2 and Table 3, an examination of these groups shows an increase in mean scores from exam 1 to exam 2 with the initial mean score for Spring 1998 being lower. Also found was a decrease in the standard deviation of those scores, with the Spring 1998 standard deviation being smaller. This is consistent with the survey findings. Although the students did not feel that the map exercise, itself, helped in their understanding of the concepts, better initial understanding was realized. Greater correlation in the Spring 1998 may be due to the type of treatment given via the concept mapping exercise. Paired Differences Comparison: Exam 2 to Exam 3 When considering paired exams 2-3 in Table 2 we see some stark contrasts as a result of increased control in the exercise. The standard deviation of the scores is smaller for Spring 1998 and continues to get smaller with subsequent exposures to the mapping exercise. Correlation for the Spring 1998 (Table 3) is greater than that of Spring 1997 (Table 4). Paired Differences Comparison: Exam 1 to Exam 3: A decrease in mean score for the Spring 1998 was seen as compared to the Spring 1997. There was also a decrease in the standard deviation from the beginning of the term toward the end. This shows a better collective understanding of the topics or concepts. Results of the above analysis are reinforced by examining the confidence intervals of the Spring 1997 (Table 3) with that of the Spring 1998 (Table 4). We see that the mapping technique has had a positive effect on student's understanding of the concepts. Furthermore, as the degree of participation on the part of the facilitator increased; a) the level of understanding of the relationships between the concepts increases, and b) the benefit, satisfaction and perceived usefulness of the maps increases, also. CONCLUSION A greater positive effect was found to exist when greater control was exercised over the explanation of the meanings underlying the concept's relationships. As an implied level of uncertainty decreased through the development of "take-off" points, we could see the level of satisfaction and use of the maps increase. Increase in satisfaction cannot be totally attributed to the technique, as the correlations suggest the existence of other variables adding to the experience's positive effect. However, an increase in mean exam scores can be attributed to increased satisfaction and use of the maps. Some limitations of this type of design can be found in the monitoring of the student's participation. Given the student having the ability for taking part in the process outside the classroom, how many actually spend what would be considered an adequate amount of time constructing the map? This can be somewhat related to whether the student's, themselves, are "self-starters". This would bring about the question of whether student's learning would increase, if the exercise would be conducted as a total in-class exercise. 241 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 TABLE 1 MQM 385 PROBLEMS IN BUSINESS: QUESTIONNAIRE COMPARISON RESULTS Question Spring 1997 Spring 1998 (n = 64) (n = 62) Did you find the use of drawing a concept map helpful in your understanding of the relationships Yes = 47% Yes = 48% of the concepts presented in the course? No = 53 % No = 52 % Did your concept map change appreciably every Yes = 55% Yes = 51% time you added to the map? No = 45% No = 49% Did referring to the map help in your understanding Yes = 24% Yes = 57% the existing relationships as new "concepts" No = 57% No = 43% are added throughout the course? Did the concept map assist you in overall in-class Yes = 24% Yes = 76% preparation? No = 76% No = 24% Did the concept map assist you in preparing Yes = 19% Yes = 70% for exams? No = 81% No = 30% TABLE 2 PAIRED T-TEST COMPARISON: SPRING 1997 - SPRING 1998 Spring 1997 n = 89 Spring 1998 n = 63 Paired Differences Paired Differences Paired Exams Mean Std. Dev Std. Error Mean Std. Dev. Std. Error 1 - 2 -9.77 13.96 1.48 -4.86 14.20 1.73 2 - 3 17.45 20.46 2.16 2.7 16.41 2.00 1 - 3 7.67 23.80 2.52 -1.94 14.38 1.75 TABLE 3 PAIRED SAMPLE STATISTICS - SPRING 1997 (n = 89) 95% Confidence Interval 2 -tail of the Difference Paired-Exams t-value Corr. Significance Lower Upper 1 - 2 -6.602 .360 .000 -12.7177 -6.8319 2 - 3 8.046 .164 .000 13.1397 21.7592 1 - 3 3.042 .094 .003 2.6615 12.6868 242 Developments in Business Simulation and Experiential Learning, Volume 26, 1999 TABLE 4 PAIRED SAMPLE STATISTICS - SPRING 1998 (n=63) 95% Confidence Interval 2-tail of the Difference Paired-Exams t-value Corr. Significance Lower Upper 1 - 2 -2.70 .485 .009 -8.1515 -1.2216 2 - 3 1.370 .367 .175 -1.2566 6.7492 1 - 3 -1.104 .300 .274 -5.4494 1.5688 REFERENCES [1] Micklich, Douglas L., The Use of Concept Mapping in Teaching Strategic Management, ABSEL Conference Proceedings, 1998 [2] Nowak, Joseph, D., and D Bob Gowin, Learning How To Learn, Cambridge University Press, New York, 1984, p. 23 [3] Trochim, William, M.K. (1989a), An Introduction To Concept Mapping for Planning and Evaluation, Evaluation and Program Planning, 12, 1, 1-16 http://trochim.human.cornell.edu/kb/conmap.htm 243 Table of Contents Volume 26, 1999 ABSEL's Historical Research Interests Back From the Future: An ABSEL Merlin Exercise for the Year 2005 The Contributions of ABSEL During the 1980's ABSEL's Contributions to Experiential Exercises in the 90's ABSEL's Contributions to Experiential learning/Experiential Exercises: The Decade of the 1970's Business Simulations - Algorithms and Model Enhancements: A 25 year Review A Study of the ETS General Field Test as an AACSB Assessment Tool and the Impact of Experiential Exercises and Simulation on Learning A Framework for Assessing the Competencies Reflected in Simulation Performance Developing A Learning Culture: Assessing Changes in Student Performance and Perception Financial Simulation Using Distributed Computing Technology Analyzing Managers' Judgements and Decisions with an Educational Business Simulation Understanding Your Business through Home-Made Simulator Development An Examination of a Reanalysis of the Impact of a Market Leader on Simulation Competitors' Strategies Applying Shocks to TE Simulations: A Demonstration Increasing Efficiency of Management Skill Assessment A Testbank for Measuring Total Enterprise Simulation Learning Developing Leadership Skills - Video Live! LEADSIMM: Collaborative Leadership Development for the Knowledge Society A Team Approach to Producing Multi-Media Laptop and Video Formatted Presentation Tinkertoys Revisited: Exploring Trust Based Relationships Overall Dominance in Total Enterprise Simulation Performance Success or Bankruptcy: The Relationship between Personal and Goal Orientation and Simulation Performance Assessing the Effects of Feedback: Muti-method and Muti-directions in Multi-pedagogical Courses The Missing Ingredients in Experiential Learning Purpose and Learning Benefits of Business Simulations: A Design and Development Perspective Building Capabilities for Change through Laboratory Simulations Modeling Innovation as a Process of Design in Educational Business Simulation The Need to Measurer variance in Experiential Learning and a New Statistic to do so Assessing Effectiveness of an Experiential Oriented Course Over Time Developing Participant Satisfaction Models of Experiential Exercises in Business Education Perceptions of Learning in TE Simulations Students' View on the Use of Business Gaming in Hong Kong Management Gaming's Lost Opportunity: Meditations about the Russian Experience and Prospects for the Future Unanticipated Enhancements in the Business Strategy and Policy Game when Running in Windows 95 Using a Business Game to Demonstrate Broad Business Concepts A New Model for Business Courses (Getting the Student Connected) Seeing the Forest and the Trees: Integrating Knowledge Using Large Scale Simulations in Capstone Business Strategy Classes Is It Here To Stay? A Roundtable Discussion of the Inter-Group Interaction Interactive Tools Used in Applying Financial Concepts Strengthening Essential Skills through a Finance Exercise: Calculation of Beta A Model of Currency Exchange Rates Understanding Currency Exchange Rates: A three-part Exercise Student Experiences in the World Intercollegiate Business Game Competition Student Experiences in the International-Collegiate Business Policy Game Competition Using Boards of Directors in Simulation Environments: Comments From Board Members Sharing Best Practices: Teaching Smarter, Not Harder Star Power: A Simulation for Understanding Power and Empowerment The Marketing Game: A Marketing Principles Simulation Alexander Islands: GSSM Tiny Business Simulator on the WWW Putting Strategy into Strategic Business Games Industry Analysis, Porter's Five Forces Model and Strategic Group Maps in the Business Strategy Game Simulation The Use of Concept Mapping to Improve Student Performance and Understanding of Strategic Management Concepts: A Comparison of Techniques Transformational thinking in the Organizational Behavior Course: The Use of Metaphor as an Assessment Tool Creation of a Virtual Learning Community for the Global Virtual Enterprise Project Inter Institutional use of Educational Resources: Joint Use of Management Simulation Games Business Insights: Theory and Practice with the Aid of a Business Simulation Progress: An Experiential Exercise in Development Marketing The ABC's of teaching the Theory of Constraints to Undergraduate Business Students Demonstrating Principles of Organizational Purchasing Behavior through an Experiential Exercise The Virtual Manager: A Different Simulation for Managing Complexity When the Rules are changing and Chaos Breeds Innovation: Recapturing the Value of Constructive Thinking and Play in Simulation Training The Pitfalls, and Potential, of Actual Events as Problem Drivers Using Business Games to teach Environmental Awareness and Green Management: The International Experience with the ENSIM Game A Review of my ABSEL-Related Work Simulation of Government: A Workshop Using GEO Creating Internet-Based Games Using Perl and JavaScript Who's on First? Exploring the Concepts of Problem-Based Learning, Experiential Learning, and Lifelong Learning Students Learn Customer Service and Selling while Conducting Research So You Want to Run an NFL Football Team–An Honors Interdisciplinary Project Supervised Internship: The Employer's Perspective Using Computer Assisted Simulation to Teach International Business Strategy: A Case of the Multinational Management Game (MMG) Multi-Cultural Experiential Learning: A Computer Simulation in China An Appreciative Stance on Diversity as We Move into the 21st Century: A Timeline Exercise to Identify Key Experiences in Good Work Relationships between Black and White Peers A Day in the Life of an Interactive, Real Time, and Internet Delivered Course: A Demonstration Comparing Internet Search Engines: An Experiential Learning Exercise Total Enterprise Simulations and the Internet: Assessing Student Perceptions and Preferences Teaching Accounting Information Systems in a Practicum Format Providing Experiential Learning in Accounting through a Field Study Payroll Project A Spreadsheet Based Business Simulation Game Computer-Behavioral Simulations Training for Project Managers Simulation Scenarios - Rationale and Illustration