21-Manuscript-197-4-11-20200213 McKim and McKendree Advancements in Agricultural Development Volume 1, Issue 1, 2020 agdevresearch.org 1. Aaron J. McKim, Assistant Professor, Michigan State University Natural Resources Building, 480 Wilson Road, Room 131, East Lansing, MI 48824 amckim@msu.edu, https://orcid.org/0000-0002-0600-3611 2. Robert “Buddy” McKendree, Assistant Professor, Michigan State University Natural Resources Building, 480 Wilson Road, Room 131, East Lansing, MI 48824 rbm@msu.edu, https://orcid.org/0000-0003-0351-777X 38 Metacognition, Systems Thinking, and Problem-Solving Ability in School-Based Agriculture, Food, and Natural Resources Education Aaron J. McKim1, R. Bud McKendree2 Abstract Problems within agriculture, food, and natural resource (AFNR) systems are increasingly complex, expanding the need for students to develop problem-solving abilities alongside an understanding of their own thinking. In this effort, we explored the problem-solving abilities, metacognition, and systems thinking of current AFNR secondary school students. A descriptive correlational study design was employed. Data were collected via an online, Qualtrics survey. Student responses on the metacognition, systems thinking, and problem-solving ability survey items suggested opportunities for increased focus on these areas throughout their educational experience. In addition, statistically significant relationships between metacognition and systems thinking and systems thinking and problem-solving ability reinforced the importance of these topics within AFNR Education. Recommendations emerging from this work include additional emphasis on metacognition, systems thinking, and problem-solving skills within secondary school AFNR classrooms; utilization of specific strategies to increase systems thinking; operationalizing specific strategies to increase metacognitive knowledge and metacognitive regulation; along with a cross-cutting recommendation for teachers to make their thinking more explicit during instruction to increase metacognition, systems thinking, and problem-solving skills among learners. Keywords Path Analysis; Transformational Classrooms; Self-Society Framework; Complex Problems; Interdisciplinary Thinking McKim and McKendree Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i1.21 39 Introduction and Problem Statement The students of today need to be prepared to solve the problems of tomorrow. In agriculture, food, and natural resources (AFNR), the problems of tomorrow (e.g., impacts of climate change, water quality, soil degradation) are increasingly complex due to the vast network of changing systems which impact, and are impacted by, AFNR (Fischer, Shah, Tubiello, & van Velhuizen, 2005). Therefore, educational systems in AFNR must provide students with opportunities to develop their own problem-solving abilities (National Academies of Science, Engineering, & Medicine, 2018). Research on the development of problem-solving skills suggests the need for transformed learning spaces, which encourage student understanding of their own thinking processes alongside understanding the dynamic systems inherent within authentic and complex problems (Augustine, 2006). Given the abundant opportunities, and necessity, for problem solving within AFNR systems, AFNR Education must expand current knowledge related to problem solving ability (National Academies of Science, Engineering, & Medicine, 2018; Sterling, 2001). In this effort, the current study explores the problem-solving abilities of current AFNR secondary school students alongside two elements of transformed learning environments, metacognition (i.e., thinking about thinking) and systems thinking (i.e., understanding the dynamic and complex nature of problems and/or phenomena). A more robust understanding of metacognition, systems thinking, and problem-solving abilities among AFNR students is expected to empower AFNR Education by identifying variables which potentially contribute to the problem-solving abilities of learners. Theoretical and Conceptual Framework The framework for this study brings together literature from three conceptual underpinnings of the current study: (a) metacognition, (b) systems thinking, and (c) problem-solving ability. Through a description of each concept, potential relationships are uncovered and a conceptual model of identified relationships is developed. Metacognition Metacognition is defined as recognizing and regulating your own thinking (Hughes, 2017). Metacognition is comprised of two distinct elements: (a) metacognitive knowledge: knowing the optimal ways you process information (Jacobs & Paris, 1987; Schraw, 1998) and (b) metacognitive regulation: actively regulating your thinking (Brown, Bransford, Ferrara, & Campione, 1983). The major challenge associated with metacognition is the “hidden” nature of thinking (National Research Council, 2000). While covert, metacognition has been found to increase learning among students (Gourgey, 1998; Pate & Miller, 2011; Wang, Haertel, & McKim and McKendree Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i1.21 40 Walberg, 1990). Similarly, metacognition has been linked to scientific processes like hypothesis generation and problem solving (Blackburn & Robinson, 2016; Pate & Miller, 2011). Systems Thinking Systems thinking refers to the “awareness of and consideration for the interaction, synchronization, and integration of people, processes, and technology among dynamic and fluctuating systems” (Stalter et al., 2016, p. 326). Systems thinkers view problems and/or phenomena from a complex, nonlinear perspective and seek to identify patterns and relationships among interdependent components of a system (Leischow & Milstein, 2006). For example, when presented with the problem of soil erosion, a systems thinker might consider the historical, political, cultural, environmental, social, and economic elements of the problem and potential solutions. Systems thinking has been linked to metacognition through the self- society framework. In essence, this framework argues awareness of thinking is required for individuals to position themselves to think about complex systems (Cabrera, Cabrera, & Powers, 2015; Gregory, 2000). Problem Solving Ability Problem solving is defined as “efforts toward achieving a situational goal for which there is no direct solution” (Pate & Miller, 2011, p. 73). Both metacognition and systems thinking have been linked to problem solving ability. Due to overlap in executive control processes, higher metacognitive skill usage has been linked to improved problem solving (Magno, 2010). Specifically, problem solving requires the ability to sift through information, distinguishing between accurate and inaccurate information, and identify relevant and irrelevant facts, actions enhanced by metacognitive processing. In addition, research suggests systems thinking is a critical skill for problem solving. Highlighting individuals able to understand the interdependency of subsystems as well as unforeseen and long-term consequences of solutions, both Checkland (1999) and Senge (2014) laud systems thinking as a critical skill in complex problem solving. The emergent relationships found between metacognition, systems thinking, and problem- solving ability comprise the conceptual model utilized within the study (see Figure 1). In this model, problem-solving ability is predicted by the combination of metacognition and systems thinking. Additionally, the model posits metacognition as a determinant of systems thinking. Problem-Solving Ability Systems Thinking Metacognition McKim and McKendree Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i1.21 41 Figure 1. Conceptual model of relationships between metacognition, systems thinking, and problem-solving ability. Purpose The purpose of this exploratory research was to analyze a conceptual model of the relationships between metacognition, systems thinking, and problem-solving ability among school-based agricultural education students. To accomplish this purpose, the following research objectives were identified: 1. Determine the metacognition, systems thinking, and problem-solving abilities perceived by secondary school AFNR education students. 2. Analyze a conceptual model of the relationships between metacognition, systems thinking, and problem-solving ability. Methods A descriptive correlational study design was employed to achieve the purpose and accomplish the stated research objectives. Data presented are part of a larger study exploring levels of student engagement in school-based AFNR education, metacognition, systems thinking, and problem-solving abilities. Target Population & Sampling The target population for this study included all secondary school students taking courses from school-based AFNR teachers in Michigan during the fall semester of the 2019-2020 school year. A stratified sample of students taught by twelve school-based agriculture teachers was purposively selected. Selected teachers were identified by engagement in a graduate level course offered by Michigan State University as well as teachers hosting micro-teaching experiences for preservice teachers at Michigan State University. Students from four of the recruited schools provided responses to the survey, yielding a 33.00% response rate among recruited teachers. In total, 56 secondary school student responses were received (n = 56). All respondents had a consent form signed by parent or guardian on file with the research team prior to completing the survey. Due to the sampling procedures, the data presented are not representative of the population. Instrumentation & Data Collection Data were collected via an online, Qualtrics survey distributed in September and October of 2019. The survey included three constructs of interest, metacognition, systems thinking, and problem-solving ability. An existing construct, the Junior Metacognitive Awareness Inventory (JrMAI), was used to measure metacognition (Sperling, Howard, Miller, & Murphy, 2001), including both metacognitive knowledge and metacognitive regulation. The JrMAI is 18 questions long (e.g., “I use different learning strategies depending on the task.”). An existing construct, the Systems Thinking Scale (STS), was used to measure systems thinking (Moore, McKim and McKendree Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i1.21 42 Dolansky, Singh, Palmieri, & Alemi, 2010). The STS includes 20 questions (e.g., “I consider how multiple changes affect each other.”). Similarly, an existing construct, Problem-Solving Ability Scale (PSAS) from the National Longitudinal Study of Adolescent to Adult Health, was used to measure problem-solving ability (Harris & Udry, 2018). The PSAS included four questions (e.g., “When I am attempting to find a solution to a problem, I usually try to think of as many different ways to approach the problem as possible.”). For consistency, each item within the three constructs was measured from 1 (never) to 5 (always). Validity and Reliability Each of the three constructs have been validated within existing literature; readers are encouraged to review the citations corresponding with each construct for more information about the validation process. To ensure the constructs were reliable among respondents, a post-hoc reliability analysis was completed. Each of the constructs fell within the acceptable range for reliability using Cronbach’s alpha (i.e., metacognition = .83; systems thinking = .93; problem-solving ability = .72; Fraenkel & Wallen, 2000; Nunnally & Bernstein, 1994). Limitations Two major limitations are considered in the current study. First, the data are limited by the honesty and truthfulness of the 56 respondents who provided data to the current study. Second, the data collection methods did not include more robust measures of metacognition (e.g., participant observations), systems thinking (e.g., mental modeling), or problem-solving ability (e.g., case problem analysis) due to feasibility, costs, and time constraints. Data Analysis Data collected for research objective one were analyzed by condensing items within the three constructs into three average construct scores, identifying the minimum and maximum response for each construct across participants, averaging each construct score across responses, and reporting the standard deviation among respondents. For objective two, a path analysis was completed in concert with the conceptual model. To complete the path analysis, two linear regressions were completed. The first regression included metacognition as the exogenous variable with systems thinking as the endogenous. The second regression included both metacognition and systems thinking as exogenous variables and problem-solving ability as the endogenous variable. Path analysis was selected as it provided a holistic evaluation of the conceptual model without requiring the sample size of more robust statistical analyses (e.g., structural equation modeling). Findings A brief description of respondents is provided in an effort to contextualize the data presented. Respondents included a larger number of females (62.75%) than males (37.25%). The largest proportion of respondents were sophomores (44.20%) followed by seniors (30.80%) and juniors (25.00%). With regard to FFA participation, the majority were non-officer members (71.20%) followed by FFA officers (19.20%) and students who were not involved in FFA at the time of McKim and McKendree Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i1.21 43 data collection (9.60%). In addition, the majority of respondents reported having a Supervised Agricultural Experience (76.90%) and being interested in a career in agriculture, food, and natural resources (70.60%). In research objective one, the metacognition, systems thinking, and problem-solving abilities perceived by students were examined (see Table 1). Students rated their problem-solving abilities (M = 3.57; SD = 0.80) slightly higher than metacognition (M = 3.55; SD = 0.64) or systems thinking (M = 3.46; SD = 0.65). A large range of responses, however, was identified for each of the three constructs. Table 1 Metacognition, Systems Thinking, and Problem-solving Ability Variables Minimum Maximum M SD Metacognition 1.06 4.78 3.55 0.64 Systems Thinking 1.85 5.00 3.46 0.65 Problem-Solving Ability 1.00 5.00 3.57 0.80 Note. Items measured from 1 (never) to 5 (always). In research objective two, the relationships between metacognition, systems thinking, and problem-solving ability were analyzed in accordance with the conceptual model (see Figure 2). The first path analyzed was the relationship between metacognition, the exogenous variable, and systems thinking, the endogenous variable. The model was statistically significant (F-value = 94.04; p-value < .001). Further, metacognition was a statistically significant predictor of systems thinking (B = 0.81, SEB = 0.08; β = .80; p-value < .001) and predicted 64% of the variance in systems thinking (r = .80, r2 = .64). Problem-Solving Ability Systems Thinking Metacognition R2 = .64 R2 = .64 .65 .17 .80 McKim and McKendree Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i1.21 44 Figure 2. Output diagram of relationships between metacognition, systems thinking, and problem-solving ability. The second path analyzed included both metacognition and systems thinking as exogenous variables with problem solving ability serving as the endogenous variable. The second model was statistically significant (F-value = 43.53; p-value < .001). In total, the combination of metacognition and systems thinking predicted 64% of the variance in problem-solving ability (r = .80, r2 = .64). Systems thinking was identified as a statistically significant predictor of problem- solving ability (B = 0.79, SEB = 0.17; β = .65; p-value < .001); however, metacognition was not a statistically significant predictor of problem-solving ability within the model (B = 0.21, SEB = 0.18; β = .17; p-value = .237). Conclusions, Discussion, and Recommendations The need for educational systems to develop learners able to solve problems compelled our analysis of problem-solving ability, metacognition, and systems thinking among school-based AFNR Education students. Findings from the study help to refine the conceptual model presented in Figure 1. The lack of a statistically significant relationship between metacognition and problem-solving ability, when accounting for systems thinking, suggests a slightly modified model (see Figure 3) in which metacognition directly influences systems thinking with systems thinking directly relating to problem solving ability. Each of the relationships proposed in the reconceptualized model were supported by the findings. Figure 3. Reconceptualized model of metacognition, systems thinking, and problem-solving ability. In addition to a new perspective of the relationships between metacognition, systems thinking, and problem-solving ability, the current study illuminated additional points of discussion. First, the average scores for metacognition (i.e., 3.55 out of a possible 5.00), systems thinking (i.e., 3.46 out of a possible 5.00), and problem-solving ability (i.e., 3.57 out of a possible 5.00) indicate opportunities for growth among responding students. Additional support for interventions to increase metacognition and systems thinking emerged from the path analysis of the relationships between constructs. Starting from the right side of the model, evidence suggests systems thinking is an important predictor of problem-solving ability. This finding is supported by existing research (Checkland, 1999; Senge, 2014). The identified importance of systems thinking suggests a greater emphasis on systems thinking within school-based AFNR classrooms could benefit the problem-solving abilities of learners. Methods for increasing systems thinking within classrooms include the use of case study analyses highlighting interdependence among systems, interdisciplinary connections being made explicit during Problem-Solving Ability Systems Thinking Metacognition McKim and McKendree Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i1.21 45 learning experiences, and encouraging student use of mental modeling to illustrate their thinking. Continuing to work left on the model, findings support a relationship between metacognition and systems thinking. The positive relationship between metacognition and systems thinking is supported by the self-society framework (Cabrera, Cabrera, & Powers, 2015; Gregory, 2000); specifically, that an individual more apt to internally consider their own thinking is positioned to think more broadly and systematically about external factors. Given the identified importance of systems thinking to problem-solving ability alongside the self-society framework, it is recommended that school-based AFNR Educators explore strategies for enhancing the metacognition of learners. Strategies for increasing metacognition include having students evaluate multiple learning strategies for their efficacy and asking metacognitive reflection questions to learners (e.g., “what about this problem is challenging,” “how are you learning this”) to build metacognitive knowledge. In addition, using a metacognitive regulatory checklist and asking metacognitive action questions of leaners (e.g., “in what other contexts could you use this learning strategy,” “what have you used successfully in the past to solve a problem like this”) are recommended strategies for building metacognitive regulation. Importantly, a simple recommendation which attends to both the need to develop systems thinking and metacognition among learners is for educators to make their thinking explicit during classroom instruction. For example, an educator who highlights why a particular answer may be more favorable when considering the interdependent systems while also articulating their thinking as they process to that conclusion has the potential to foster both systems thinking and metacognition among learners. In addition to practical recommendations for increasing metacognition, systems thinking, and problem-solving ability among secondary school learners, authors recommend more rigorous analyses of the reconceptualized model. As a roadmap, evaluating metacognition using regulatory checklists and participant observations, systems thinking using mental models, and problem-solving abilities using case studies will provide a more in-depth understanding of the relationships supported by this exploratory research. Further, obtaining a large enough sample to evaluate the model using structure equation modeling, in which systems thinking simultaneously acts as an endogenous and exogenous variable, will address the error variance unaccounted for within the path analysis approach. Future research can build upon the foundation provided by the exploratory study to uncover additional insights into the relationships between these important educational outcomes. With the opportunity to more effectively prepare the learners of today to be problem-solving leaders of tomorrow, analyses and interventions to understand and increase metacognition, systems thinking, and problem-solving skills within AFNR Education are essential to addressing the increasingly complex challenges facing the world. McKim and McKendree Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i1.21 46 References Augustine, N. (2006). Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington, D.C.: National Academy Press. Blackburn, J. J., & Robinson, J. S. (2016). Determining the effects of cognitive style, problem complexity, and hypothesis generation on the problem-solving ability of school-based agricultural education students. Journal of Agricultural Education, 57(2), 46-59. https://doi.org10.5032/jae.2016.02046 Brown, A., Bransford, L., Ferrara, R., & Campione, J. (1983). Learning, remembering and understanding. In P. H. Mussen (Ed.), Handbook of child psychology (4th ed., pp. 77). New York, NY: John Wiley and Sons. Cabrera, D., Cabrera, L., & Powers, E. (2015). A unifying theory of systems thinking with psychosocial applications. Systems Research and Behavioral Science, 32, 534-545. https://doi.org/10.1002/sres.2351 Checkland, P. B. (1999). Systems thinking, systems practice. New York, NY: John Wiley and Sons. Fischer, G., Shah, M., Tubiello, F. N., & van Velhuizen, H. (2005). Socio-economic and climate change impacts on agriculture: An integrated assessment, 1990-2080. Philosophical Transactions of the Royal Society London B: Biological Sciences, 360(1463), 2067-2083. Fraenkel, J. R., & Wallen, N. E. (2000). How to design and evaluate research in education. New York, NY: McGraw-Hill. Gourgey, A. F. (1998). Metacognition in basic skills instruction. Instructional Science, 26(1), 81- 96. Gregory, W. J. (2000). Transforming self and society: A “critical appreciation” model. Systemic Practice and Action Research, 13(4), 475-501. Harris, K. M., & Udry, J. R. (2018). National longitudinal study of adolescent to adult health 1994-2008. Ann Arbor, MI: Carolina Population Center, University of North Carolina- Chapel Hill. https://doi.org/10.3886/ICPSR21600.v21 Hughes, A. J. (2017). Educational complexity and professional development: Teachers’ need for metacognitive awareness. Journal of Technology Education, 29(1), 25-44. https://doi.org/10.21061/jte.v29i1.a.2 Jacobs, J. E., & Paris, S. G. (1987). Children's metacognition about reading: Issues in definition, measurement, and instruction. Educational Psychologist, 22(3-4), 255-278. Leischow, S. J., & Milstein, B. (2006). Systems thinking and modeling for public health practice. American Journal of Public Health, 96(3), 403-405. https://doi.org/10.2105/AJPH.2005.082842 Magno, C. (2010). The role of metacognitive skills in developing critical thinking. Metacognition and Learning, 5(2), 137-156. McKim and McKendree Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i1.21 47 Moore, S. M., Dolanksy, M. A., Singh, M., Palmieri, P., & Alemi, F. (2010). The systems thinking scale. Cleveland, OH: Case Western Reserve University. National Academies of Sciences, Engineering, and Medicine. (2018). Science breakthroughs to advance food and agricultural research by 2030. Washington, DC: The National Academies Press. https://doi.org/10.17226/25059 National Research Council. (2000). How people learn. Washington, D.C.: National Academy Press. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw- Hill. Pate, M. L., & Miller, G. (2011). Effects of regulatory self-questioning on secondary-level students' problem-solving performance. Journal of Agricultural Education, 52(1), 72-84. https://doi.org/10.5032/jae.2011.01072 Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26(1), 113-125. Senge, P. M. (2014). The fifth discipline fieldbook: Strategies and tools for building a learning organization. New York, NY: Crown Business. Stalter, A. M., Phillips, J. M., Ruggiero, J. S., Scardaville, D. L., Merriam, D., … Winegardner, S. (2016). A concept analysis of systems thinking. Nursing Forum, 52(4), 323-330. Sperling, R. A., Howard, B. C., Miller, L. A., & Murphy C. (2002). Measures of children’s knowledge and regulation of cognition. Contemporary Educational Psychology, 27, 51- 79. Sterling, S. (2001) Sustainable education. Re-visioning learning and change (Bristol, J. W. Arrowsmith) (Schumacher Briefings, 6). Wang, M. C., Haertel, G. D., & Walberg, H. J. (1990). What influences learning? A content analysis of review literature. The Journal of Educational Research, 84(1), 30-43. © 2020 by authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).