Can Systemic Thinking be Measured? Introducing the Systemic Thinking Scale Page 151 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 ABSTRACT This paper presents an exploratory study to measure systemic thinking in undergraduate business students. Based on litera- ture related to cognitive styles, we identify three dimensions of systemic thinking - locus of attention, inter-relatedness and flex- ibility. The authors developed a 25-item Systemic Thinking Scale (STS) which was tested on undergraduate students during Fall 2014. The scale's validity and reliability will be examined and results presented at ABSEL’s 2015 meeting. INTRODUCTION Management education is tasked with developing decision makers that can manage in global and, sometimes, turbulent environments. Given that managers coordinate people, ideas, and beliefs in implementing strategies to achieve an organiza- tion’s goals, how these managers think will likely play a role in their decision making process. Systemic thinking, an approach to understanding reality, recognizes that systems have charac- teristics and patterns independent of their parts. This supports the recommendation that business leaders develop skills that strengthen their ability to view an entity in a holistic way (Allio, 2003). An exploratory study by Washington, Kurthakoti, Halpin, & Byrd (2014) measured the change in the level of systemic thinking of students using a total enterprise business simulation. Using a rubric developed for the study, the researchers complet- ed a content analysis of statements made by decision makers in an early and then later stage of the simulation. Results showed an increase in the systemic thinking skills of students as they progressed in this exercise. In addition, higher levels of system- ic thinking in early periods of the simulation were positively related to subsequent performance (p < 0.10). Although insight- ful as to the importance and impact of systemic thinking on performance, we feel this study has certain limitations. First, the rubric developed for the study was specific to the course and the simulation being used. This limits the findings to be applied to courses using other simulations. Second, while the rubric de- veloped may be useful for assessing systemic thinking (with adjustments if necessary) in a simulation environment it is not an appropriate methodology for assessing this skill across dif- ferent pedagogical approaches to teaching business concepts (lectures, experiential exercises, and cases, and simulations). The current study aims to address these issues by developing a comprehensive scale to assess systemic thinking across all methods of instruction. Once tested for validity and reliability, we believe that the scale can be used across disciplines and ped- agogical approaches. LITERATURE REVIEW “Systems thinking is a way of understanding reality that emphasizes the relationship among various components in a process, rather than the independent constituents of the pro- cess” (Gregory & Miller, 2011, p. 259). Recognizing that a sys- tem has characteristics and patterns independent of its parts provides a rationale for business leaders to acquire skills that draw on their ability to view an entity in a holistic way (Allio, 2003; Henning & Chen, 2012). Systemic thinking integrates analysis and synthesis and is said to lead to greater understand- ing and better decision making. According to Laszlo (2012), “Analysis answers the questions ‘what’ and ‘how’…Synthesis answers the ‘why’ and ‘what for’ questions” (p. 97). Research on individual cognitive styles provides a starting point for us to identify the key dimensions of systemic thinking. How one organizes and processes information is known as one’s cognitive style. When applied to how one completes a task or responds in a decision making situation, some individu- als may focus on the individual parts of the task while others take the set of information and process it in a global context. These different ways of thinking are thought to be relevant in problem solving situations and may help predict the success rates of decision makers (Sadler-Smith & Badger, 1998). Stern- berg & Wagner (1998) offered thirteen thinking styles and cre- ated a scale, the Mental Self-Government Thinking Styles In- ventory (MSG), which attempted to categorize individuals based on how they approach problem solving situations. They identified a number of tendencies in decision makers such as rule making, goal setting, and flexibility. Choi, Koo, & Choi (2007) contributed further to our under- standing of thinking styles with their Analysis-Holism Scale, which distinguished between individuals who view the world in a holistic way and those with a focus on the world as a set of independent components. One of the four domains examined in this work, locus of attention, is a dimension of the construct which is the focus of the proposed study on systemic thinking. CAN SYSTEMIC THINKING BE MEASURED? INTRODUCING THE SYSTEMIC THINKING SCALE (STS) Annette L. Halpin Arcadia University halpina@arcadia.edu Raghu Kurthakoti Arcadia University kurthakotir@arcadia.edu mailto:halpina@arcadia.edu mailto:kurthakotir@arcadia.edu Page 152 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 Business schools and programs are expected to graduate professionals with the knowledge and skills to manage in a global economy. Facing increasingly complex environmental factors, solutions to problems today are neither apparent nor satisfactory (Caldwell, 2012). In the current business climate, decision makers must arrive at solutions in shorter periods of time and with less than complete information (Noel & Erskine, 2013). Leaders who are systemic thinkers will be able to make sense of situations where patterns of behavior and relationships did not exist previously and exhibit higher-order thinking which includes analysis, synthesis, and evaluation as defined by Bloom, Englehart, Furst, Hill, & Krathwohl ( 1959) or, more recently, the ability to analyze, evaluate, and create as offered by Krathwohl (2002).From the perspective of management edu- cators, identifying which teaching methods strengthen the sys- temic thinking skills of students is a notable goal. How to meas- ure this is the challenge for researchers. The current research proposes the use of a survey instru- ment, the Systemic Thinking Scale (STS) to assess systemic thinking in individuals. The items on the scale capture the three dimensions the authors propose are the components of this con- struct. Included are the following. Locus of attention: This dimension pertains to what a deci- sion maker focuses on while making decisions. Some individu- als focus on the parts of a task when developing a response while others take the information and process it all within a holistic context (Sadler-Smith & Badger, 1998). Systemic thinking requires a more holistic approach - an ability to view the whole task with qualities and characteristics distinct from its parts. In two studies using a business simulation as a tool there were differences in holistic cognitive perceptions across indus- tries (Wellington, Faria, & Whiteley, 1998) and a persistent positive relationship between systemic thinking and perfor- mance over several periods (Washington, et.al., 2014). This dimension is captured in the STS inventory through various statements that assess what the students pay attention to while performing tasks. (E.g. When working on a task, I like when I need to pay attention to details; I like tasks where I can focus on general ideas, rather than specifics.) Inter-relatedness: Successful decision making by managers requires an appreciation of the interconnectedness of the parts of a task or issue. Knowledge of theories and models is part of the dimension of ‘Conceptual Knowledge’ that Krathwohl (2002) suggests enables a learner to understand how a system functions. A total enterprise simulation provides numerous op- portunities for individuals to test their decision making skills. Individuals respond each period to circumstances shaped by their own decisions as well as those of other firms in the indus- try. Students must consider how marketing, operations, finance, and management decisions affect one another as well as the total set of decisions influences the overall performance of the organization. This dimension is captured in the STS inventory through various statements that assess how the students per- ceive the relationship between the whole and the parts. (E.g. When working on a task, I like to see how what I do fits into the overall picture; Everything associated with a task is somehow related to each other.) Flexibility: In addition to thinking in a holistic way that sees the relationship between various components of a task/ problem, a systemic thinker should also be creative and respon- sive to changing conditions while solving problems. This means having the ability and the will to be flexible in one’s approach to problem solving. As organizations strive to remain relevant, successful leaders often frame the situations they face in a way that allows for flexibility (Zaccaro, Gilbert, Thor, & Mumford, 1991). It is not uncommon for experienced decision makers to develop a range of appropriate responses to a given problem. These experiences or ‘scripts’ are acquired over time and stored to memory – to be recalled at a later date. This increases the flexibility of a manager’s decision making style and can lead to a greater ability to be creative in solving problems (Gioia & Poole, 1984). It is not the routine decision making situation that requires flexibility so much as the atypical one which calls on a leader to search through his/her array of past experiences to arrive at a range of possible solutions. Thus, we believe that a scale measuring systemic thinking should include a third di- mension we call ‘flexibility’. This component is captured in the STS inventory through various statements that assess students’ extent of rigidity in completing tasks. (E.g. When considering ways to complete a task, I tend to approach it in a traditional way; When working on a task, I like to do things in new ways not used by others in the past). Understanding the thinking process of decision makers may help explain the quality of the solutions they offer and whether there is any relationship between thinking style and performance (Gioia & Poole, 1984). To that end, the goal of the proposed research is to identify the level of systemic thinking of individuals prior to and after the use of a simulation and to re- late the level of systemic thinking to performance on a total enterprise computer simulation. The Systemic Thinking Scale (STS) measures the three dimensions of systemic thinking - locus of attention, task interrelatedness, and flexibility. Based on prior research, the following propositions are offered with testing to be completed during the current period (Fall 2014) and presented to the ABSEL community in Spring 2015. Proposition 1a: Students running a total enterprise business simulation will score relatively low on the systemic thinking scale before using the simu- lation. Proposition 1b: Students running a total enterprise business simulation will score relatively high on the systemic thinking scale after using the simula- tion. Proposition 2: The systemic thinking score on the STS in- ventory will be positively related to student performance on the simulation. METHODOLOGY AND DATA ANALYSIS In this study we assess the systemic thinking skills of un- dergraduates running a total enterprise simulation in an intro- ductory business course. Students were enrolled in a small lib- eral arts university located in northeastern US. In the first phase of the study, we identified and developed a 25-item Systemic Thinking Scale (STS) that we believe cap- tures the three dimensions of systemic thinking based on past studies as well as face validity. Page 153 - Developments in Business Simulation and Experiential Learning, volume 42, 2015 In the second phase this inventory was administered to about 100 student subjects who responded to each statement using a 5–point Likert Scale (Strongly Disagree to Strongly Agree). The scale was administered twice – before they began using the simulation and after they had completed the simula- tion. Unique identifiers were used to ensure matching between each subject’s “before” and “after” scores on the STS. The data is currently being analyzed to determine factor structure and validity of dimensions through reliability analysis and factor analysis. RESULTS, DISCUSSION, AND CONCLUSION The results of the analyses will be shared and discussed at the ABSEL 2015 meeting. REFERENCES Allio, R.J. (2003). Russell I. Ackoff, iconoclastic management authority, advocates a “systemic” approach to innova- tion. Strategy and Leadership, 31 (3), 19-26. Bloom, B.S., Englehart, M.D., Furst, E.D., Hill, W.H., & Krath- wohl, D.R. (1959). Taxonomy of educational out- comes: The classification of educational goals. Hand- book 1: Cognitive Domain, New York, NY: David McCay Company, Inc. Caldwell, R. (2012). Systems thinking, organizational change and agency: A practice theory critique of Senge’s Learning Organization. Journal of Change Manage- ment, 12 (12), 145-164. Choi, I., Koo, M., & Choi, J.A. (2007). Individual differences in analytic versus holistic thinking. Personality and Social Psychology Bulletin, 33, 691-705. Gioia, D.A. & Poole, P.P. (1984). Scripts in organizational be- havior. The Academy of Management Review, 9, 449- 459. Gregory, A. & Miller, S. (2011). Waving or drowning? Re- evaluating the place of systems thinking in business and management learning and curricula. Presented at the Organisational Learning, Knowledge and Capabili- ties Conference, Hull, UK. Henning, P.B. & Chen, W-C. (2012). Systems thinking: Com- mon ground or untapped territory? Systems Research and Behavioral Science, (29), 470-483. Krathwohl, D.R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory Into Practice, 41 (4), 212-218. Laszlo, K.C. (2012). From systems thinking to systems being: The embodiment of evolutionary leadership. Journal of Organizationl Transformation & Social change, 9 (2), 95-108. Noel, T & Erskine, L. (2013). The silent story: using computer- aided text analysis to predict entrepreneurial perfor- mance. Journal of Entrepreneurship, 22(1), 1-14. Sadler-Smith, E. & Badger, B. (1998). Cognitive style, learning and innovation. Technology Analysis & Strategic Management 10 (2), 247-265. Sternberg, R.J. & Wagner, R.K. (1991). Mental self government thinking styles inventory manual. Unpublished manual. Washington, M.L., Kurthakoti, R., Halpin, A.L., & Byrd. S. (2014). Assessing systemic thinking in undergraduates: An exploratory study using a total enterprise business simulation. Developments in Business Simulation and Experiential Learning, 41, 53-63. Wellington, W., Faria, A.J., & Whiteley, T.R. (1998). Holistic cognitive strategy in a computer-based marketing sim- ulation game: An investigation of attitudes towards the decision-making process. Developments in Business Simulation and Experiential Learning, 25, 246-252. Zaccaro, S.J., Gilbert, J.A., Thor, K.K., & Mumford, M.D. (1991). Leadership and social intelligence: Linking social perspectiveness and behavioral flexibility to leader effectiveness. Leadership Quarterly, 2(4), 317- 342.