Teaching Complexity: The Limits of Evidence and the ‘Prospective’ Case Study Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 92 University of Bamberg Press Teaching Complexity: The Limits of Evidence and the ‘Prospective’ Case Study Amanda Wolf Abstract: Policy practitioner–students in a Master of Public Policy programme in New Zealand describe many problems and processes in their work environments as ‘complex’. Yet, they hold firmly to a belief in the merits of ‘evidence’ to guide their advice and decision making in the face of that complexity. This article examines the aims and pedagogy of a two-course sequence designed to help students replace over-reliance on analysing existing evidence with understanding of the ways complexity concepts can aid in estimating possible outcomes of policy interventions. Starting with identifying evidence challenges, students learn how to compare a status quo situation and a prospective case of that status quo in which a new policy has been implemented. This method draws on existing scholarship in lesson-drawing for policy applications. Students are eased into an appreciation of a variety of complexity frameworks and concepts by looking at a case about which there is, strictly speaking, no evidence. Keywords: case study; evidence; lesson-drawing; New Zealand; public policy; teaching complexity Introduction Students in Victoria University of Wellington’s School of Government Master’s programme in public policy and public management are primarily working professionals who return to university for study. The programme brings students of both policy and management together for a greater part of their learning, recognising the interdependence and integration of the two fields in practice. Students in both degrees share three core courses (out of 12 required for the degree) that cover the overall context, institutions and dynamics of government and governing, national and international societal and economic forces that influence the essential features and interdependence of management and policy, and theories and practices in policy and management. The common core courses also recognise that practitioners need to be able to critically reflect on their specific practices and to have high-level intellectual capability to be both critical and creative in their work. As designed, the programme ensures that classrooms have a great diversity of learners. The courses are interdisciplinary and problem-focused, drawing on selected examples and—importantly—ongoing challenges in students’ workplaces. The explicit intentions are to increase students’ understanding of the relevance of many disciplines to their work, and to sensitise them to the challenges and expectations on them as professionals in this environment. Much of the learning in these courses is relevant for advancing students’ understanding of the complexities in policy development and implementation, but ‘complexity’ is not directly addressed as a theme. Nevertheless, from the start, students learn in an environment in which they are assisted to look at their practice, in primarily policy or management, openly and reflectively. Amanda Wolf, School of Government, Victoria University of Wellington, Wellington, New Zealand, E-mail: amanda.wolf@vuw.ac.nz http://dx.doi.org/10.20377/cgn-48 Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 93 University of Bamberg Press In addition to the common core courses, students pursue a four-course core (comprising one-third of the degree) specific to management or policy, and complete their degree with shared electives and independent research. Students in the Master of Public Policy stream study a two- course sequence, which I teach with contributions from colleagues. Teaching complexity in these two courses is the focus of this article, with a particular emphasis on using the device of a ‘prospective case’. In the next section, the policy cohort and their initial orientation to study are more fully described. Following this, my approach to teaching complexity is described, with reference to the course learning objectives, content and pedagogy, including attention to the rationale for using a prospective-case method. A final section presents some reflections on the methods used and their merits in teaching complexity to a cohort of experienced policy professionals. Master of Public Policy Students’ Initial Orientations to ‘Evidence’ and ‘Complexity’ In their workplaces, policy students are exposed to strong rhetoric and prescriptions concerning the supremacy of ‘evidence’ for developing and implementing policy. Consequently, they are intensely distrustful of any information that looks ‘subjective’ or ‘biased’. They applaud claims such as one in a report from the US National Research Council: “[W]hen the question on the table is what are the ‘real’ conditions or what will ‘probably’ happen if we implement one policy instead of another, science is on balance a more dependable and defensible guide than informed hunches, analogies, or personal experience” (Prewitt, Schwandt, & Straf, 2012, p. 10). Closer, to home, they may be employed by a departmental chief executive, who proclaims: “We have to make sure what looks like a good policy idea is backed up by solid evidence and quality analysis”. In workplace discourse, it is equally broadly claimed that policy formation and implementation is ‘complex’. Usually this refers to the presence of uncertainty and value clashes, with many inputs and actors at the interface between policy analysis and politics. And while there may be talk of ‘interdependencies’ or ‘unpredictability’—terms that are used in some complexity literature—the typical response is to seek more or better evidence, as the then head of Australia’s Productivity Commission urged: Without evidence, policy makers must fall back on intuition, ideology, or conventional wisdom—or, at best, theory alone... . But the resulting policies can go seriously astray, given the complexities and interdependencies in our society and economy, and the unpredictability of people’s reactions to change... . policies that haven’t been informed by good evidence and analysis fall more easily prey to the ‘Law of Unintended Consequences’ which can lead to costly mistakes. (Banks, 2009, pp. 4–5) Thus, the way many students appreciate their work environment creates a significant block to introducing a deeper understanding of ‘complexity’. Accordingly, I adopt a well- established pedagogical practice, which holds that for students to learn a new concept or skill, the teacher must first activate some doubt or discomfort with the students’ pre-existing beliefs, then present new material, provide examples, and offer students an opportunity to practice applying the material. Over the two courses, I aim to help students enlarge their skills and to understand that (a) there are significant limits to using evidence (in part since it is by definition about the past); (b) they already compensate for these limits using various natural, judgemental (and hence, subjective and biased!) processes that help them to see a little bit into the future; but (c) they don’t have to give up entirely on the rigour and reliability promised by evidence; since (d) skills for working with complexity can complement what they gain from conventional evidence analysis. http://dx.doi.org/10.20377/cgn-48 Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 94 University of Bamberg Press Learning Objectives and a Stepped Introduction to Complexity Most students undertake study in economics and an introduction to policy analysis before enrolling in the remaining two courses in the policy core. The course prescriptions for the two subsequent courses refer directly to complexity: Policy Methods and Practice: Qualitative and quantitative techniques for collecting, analysing, interpreting and applying information and evidence to advance policy objectives particularly under conditions of complexity and uncertainty and in light of given task requirements. Policy Workshop: Integrating theories and practices to produce in-depth studies of selected complex policy issues with a focus on the design and evaluation of alternative policy options. In the Policy Methods course, the specific learning objectives are framed in a way that accepts that learners (and often their employers) are interested in gaining ‘conventional’ evidence skills, such as determining the specific evidence requirements for a given policy research or analysis task; selecting or adapting methods and practices for various challenges; demonstrating critical awareness of the strengths and limitations entailed in applications; and judging the adequacy of evidence. Consideration of complexity, however, begins in the first session. The first exercise is a round-robin to elicit qualities of a ‘good policy professional’, designed to activate some scepticism about the reach of ‘evidence’ in practice. As expected, everyone knows some analytical whiz, and first-round propositions are heavily weighted toward analytic skills, such as the ability to define the problem, to find and assess information or to conduct cost–benefit analyses. Soon come ideas about relationships in the environment, such as dealing with political constraints, attending to international norms, and communicating with different demographic groups or other professionals. Eventually, a few students venture some qualities consistent with managing complexity and uncertainty. For example, they mention making sense of a messy situation, or intuiting the inclinations and likely responses of a politician and other stakeholders. Often, a student will also volunteer characteristics that seem more directly to acknowledge complexity, such as the ability to anticipate or be alert to ‘unintended consequences’ or to the unpredictability of people’s behaviour. However, in offering these suggestions, students may be looking through the prism of an ‘evidence’ or analytic frame, consistent with the Banks quotation above. By the end of the round-robin, students are more attuned to some tension in their use of ‘evidence’ and ready to consider my initial definition of complexity: as a condition in which the knowledge required to bring about a desired change is not readily mustered by even the most sophisticated analysis of evidence, but is rather highly distributed in the communities, organizations and individuals where change is to take place and is unknowable at the point when policy decisions are being made. Although students arrive wanting to become better evidence-handlers, the exercise forces a small wedge into the door to complexity learning. Students realise for themselves that there is more to their practice than developing evidence products, such as analyses and ‘objective’ advice. With the scene thus established, student examine more carefully the assumption of the ‘science’ framing of evidence, drawing on arguments for the variety of information that aids decision makers (Head, 2008), the connection between complexity and the tradition of American pragmatism (Sanderson, 2009) and the distinction between information and evidence, as described by Majone (1989, p, 7): Evidence is “information selected from the available stock and introduced at a specific point in the argument in order to persuade a particular audience of the truth or falsity of a statement”. Students consider that for many policy challenges, there are unfillable gaps in the evidence base. In their first written assignment, students map out some of the ‘evidence challenges’ in a relatively discrete topic of their choosing, taking into account value differences, and various contextual features. http://dx.doi.org/10.20377/cgn-48 Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 95 University of Bamberg Press Hard on the heels of these efforts to call evidence into question, the class considers definitions of uncertainty, ambiguity and complexity and is introduced to methods and practices to address them. I present three ‘complexity frameworks’ (see Table 1), each of which takes a different tack: Eppel’s lens (2012), developed from the complexity literature, Pawson’s (2012) suggestions for ‘mapping’ a pending policy intervention, and Room’s (2011) advice to an ‘agile’ policy professional. I illustrate aspects of these frameworks using a published case study, such as Smith and Lawrence (2014) on food security and resilience in Rockhampton, Australia, following flooding that cut off a community’s access to food. Such case studies are ubiquitous in the published literature. True to form, such articles have some theoretical point to advance, and it is this that makes them generally seem ‘irrelevant’ or ‘overly academic’ to our cohort of students. Yet, almost all such articles contain detailed descriptions of a case, which can be read on their own, and are generally suited to get the reader’s complexity thinking going, with complexity frameworks providing direction and structure: What systems interacted? How did prior changes in the regulatory system change the local response to the drought? What information could have been known about the risks to the supply chain but was available only in retrospect? Finally, the students read Mary Schmidt’s (1993) story of the failure of the Teton Dam to further sensitise them to the alternative forms of knowledge potentially available to for decision-making, but which are generally dismissed from the ‘evidence’ base. Students are allocated one ‘theme’ from Table 1 or from Schmidt’s types of knowledge, which they consider with respect to the policy topic they are investigating. Class presentations ensure exposure to varied applications. The rest of the Policy Methods course covers evidence- handling skills through units on various methods of collecting, analysing and interpreting evidence and assessing its qualities in specific applications. This treatment reinforces the merits of good analysis, which is not obviated by complexity. Learning Complexity through Prospective Case Analysis In the Policy Workshop, students learn to draw and apply transferable lessons from an exemplar (Wolf & Baehler, 2017). They then undertake a realistic, if somewhat attenuated project that demonstrates their ability to apply these methods, as well as other learning from throughout the policy core. Building on the Policy Methods course (which activates doubt about the full rhetorical impact of the ‘evidence-based policy’ mantra, awakens recognition of some natural aptitude for complexity thinking, and offers exposure to numerous examples of how aspects of complexity may influence policy analysis and decision making), I continue in the Workshop to gradually draw students along a learning pathway in a structured manner. The high-level tasks in the Policy Workshop start with a status quo ‘situation analysis’ of a current policy challenge, which can be ‘cased’ as a policy status quo, or receiving environment (or ‘target case’ in the policy transfer and learning literature) or simply A, with attention to the stakeholders, problem- and solution-framings, key contextual influences and tensions among them. Next, students make a detailed study of a promising policy intervention that is absent in A, but in place elsewhere (B). By designing an intervention adapted from B for application in the receiving environment (A), students create a prospective case (A′), which is A at time t + 1, in which policy based on insights spawned by B is producing impacts, for better or worse. Such prospective-case thinking aids students in taking a broader perspective—to think about the policy ecosystem, comprising actors and resources and, importantly, other policies, and to imagine the introduction of new decisions. In sum, to estimate the impacts in A′, students take into account— at some level and selectively—aspects of the complexity frameworks in Table 1. http://dx.doi.org/10.20377/cgn-48 Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 96 University of Bamberg Press Table 1. Complexity Frameworks Eppel Complexity Lens Pawson VICTORE Model Room Agile Policymaking Toolkit The system whole – Whole cannot be understood as the sum of its parts, or reduced to its parts Nested, interacting and interdependent systems – Characteristics identified at one level of the system are also present in whole Multiple interactive systems – Reflexive influence patterns, which arise from the ongoing interaction between actors, can result in negative or positive feedback loops Adaptation and co-evolution – Changes in the environment may stimulate not only system change, but also external change in response Change through self-organisation and emergence – Change in systems occurs as a result of changes which affect the micro- diversity that emerges as a result of feedback loops, adaptation and emergence Open systems and socially constructed boundaries – Individuals can plan their own actions, not actions of others or their interplay Stability is not equilibrium – Systems can suddenly and unpredictably undergo large changes of trajectory History influences starting point for change Volitions – Participants’ motivations and reasoning; interventions don’t work; people’s interpretations and actions do Implementation – Many hands and long implementation chains; long program processes (many decision-making points) Contexts – Multiple, open system, interacting; Stakeholders, relationships, institutional settings, norms, values; culture, history, and so on Time – Historicity; sequences of programs; duration Outcomes – Short, medium and long term; multiple indicators; multiple interpretation of findings Rivalry – Multiple policies and programs operating: competing, undermining, reinforcing… Emergence – Elements interact to create new elements of systems; changes based on changes can go in any direction Map the landscape – Understand the policy arena’s issues and current challenges Identify the protagonists – Know the players, stakeholders, relationships in policy arena Model the struggle – Create scenarios to understand how the landscape may evolve Watch for tipping points – Identify the triggers that could dramatically shift matters Tune the landscape – Use analytical tools and discussions to move the policy arena into directions that are more productive Energize the protagonists – Help some of the protagonists build capacity and take other actions to encourage cooperative behaviour toward win-win situations Civilize the struggle – Help create win-win situations and limit destructive behaviours Watch for predators – Keep one or more protagonists from unfairly tipping the balance of power and creating a destructive struggle in the landscape Sources: Abstracted from Eppel (2012), Pawson, (2013), and Room (2011) http://dx.doi.org/10.20377/cgn-48 Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 97 University of Bamberg Press In the classroom, time is taken to motivate the reasons as I see them for taking a comparative-case approach and to justify the place of these methods in a complexity-ready toolkit. As writers have made clear (for example, Thomas 2010, 2011, rehearsing the arguments of MacIntyre), for one reason or another, generalisation does not work easily in the social sciences, which is a problem for the application of evidence to new decisions. No two particular instances of a policy are ever similar enough, and the differences always cast doubt on the applicability of the generalisation. While replicating studies in many different locations under a diversity of circumstances can help reduce problems of external validity, the high costs and long timeframes associated with repeating sophisticated studies make this prohibitive most of the time. Even with replication, problems of external validity tend to persist because social problems, implemented solutions, and contextual settings vary in so many different ways from case to case. In the Workshop, following their situation analyses, students find an exemplar elsewhere, assess it to gain an understanding of that other case, then abstract and adapt learning for application in the presenting case. Students read Rose (2002) and Barzelay (2007), which offer methods showing how practitioners and researchers can use existing policies or management processes as cases for learning, based on the congruence between studied cases and prospective decisions. Importantly, these methods are intended for application within a causal-analytic frame, and so they do not entail students making a cognitive alteration for which they are philosophically disinclined. ‘Lesson-drawing’ and ‘vicarious learning’ methods allow for the transportability and adaptation of “fresh thoughts” (Rose, 2005), based on a causal model of the ‘known’ and the ability of policy learners to “use their minds” (Barzelay, 2007) to make an effective transfer. Nevertheless, some discussion in the classroom gently critiques these two sources, providing students with a critical awareness of the challenges complexity offers in the search for ‘fresh ideas’ and the judgemental elements of ‘using one’s mind’ in the course of deriving and transferring lessons (Wolf & Baehler, 2017). These include the manner in which relevant fresh thoughts are generated and detected; the enormous number of variables that differentiate cases in unknown ways; and the fact that lessons are from the past but for the future, all of which complicate using one’s mind in any causal-lesson-drawing activity. While students are encouraged to undertake some analysis (for example to estimate costs of implementing the initiative in the new context based on the experience elsewhere), they have little time for actual analysis, and thus they are (intentionally) thrown into the need to make assumptions and judgemental leaps, to make use of analogical thinking, to fall back on guessing. Additional learning exemplars are discussed in class, and guests also present some ‘real’ cases involving policy transfer. For example, the Wellington City Council Community Services Manager has been invited to explain the city’s homelessness strategy. Wellington follows a ‘housing first’ strategy, based on some overseas experiences. This compares with several Australian cities that have followed a variant of this strategy, known as ‘Common Ground’ (but for which there is scant ‘evidence’; see Parsell, Fitzpatrick, & Busch-Geertsema, 2014). I look for published case studies on policy issues that are socially embedded (rather than concerning a technical matter). The implementation of Stockholm’s congestion charge (a good overview is Börjesson, Eliasson, Hugosson, & Brundell-Freij, 2013), for example, can be used to think about a prospective case in Auckland, which is also searching for ways to address congestion and faces the sort of resistance documented in Stockholm. Another example (Steyaert et al., 2007) is used to demonstrate the manner of ‘judgemental leaping’ from the presented case, which is very specific, to a very different prospective case. The authors narrate an example of community-driven ecological restoration of marshlands along the French Atlantic coast. Maraîchine cows, a local breed, had lost favour during agricultural intensification because of their low milk- and beef-production levels. A farmer fond of local history and marsh heritage began to look for the last Maraîchine specimens and to work towards http://dx.doi.org/10.20377/cgn-48 Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 98 University of Bamberg Press rebuilding the population by identifying individual cows, finding farms to house them, organising breeding and so on. In the process, the farmer brought together agricultural schools and ecological museums, but also some regular landowners and farmers. This network of people shared the objective of preserving the marshland’s biological resources. The lesson from this example relates to preserving species and diversity while also providing for community resilience: the case details (French cows on the Atlantic coast) matter less than the somewhat more abstract storyline. Making a connection requires someone (role- played by students) with an actual community-resilience challenge in mind to generate incipient hypotheses (“X here functions like the farmer there”). As further elaborated below, readers act as if they are engaging in a hypothetical dialogue with a scene that reading helps them to visualise and which in turn allows for the emergence of “pursuit-worthy” ideas (McKaughan, 2008) for application in the home environment. Reading such cases without the theory parts (like reading a regression study while skipping the numbers) can inspire fresh ideas (uninfluenced by the author’s theoretical objectives). Time is taken to consider what makes a ‘case’ and how a case may be complex. I use an ecosystem metaphor to draw attention to interactive systems and feedback loops, emergence, adaptation and co-evolution. I also reflect that cases tend to be relayed in narrative form, which helps to overcome static and single-perspective framings. I emphasise that in policy, complexity is typically not found in statements of ‘problems’ or ‘solutions’, most of which appear entirely straightforward—such as “children consume too much sugar so we need tax sugar-sweetened beverages”. Rather, it is the interactions in the world—among varied understandings of the issues, interpretations of the evidence and values—that make successful policy implementation so challenging, and that moreover, these interactions are not just hard to analyse, they are complex and cannot be known or estimated or modelled well at all. Instead of trying to overcome these obstacles with the brute force of analytic thinking, students are invited to try to relax that tendency in order to approach cases simply as stories— tales of events and actors and outcomes in unique contexts, since complexity renders such cases unknowable analytically. Yet, a policy or management practitioner, with some relevant experience, can observe and gain insights using a combination of abduction and phronesis. Briefly, abduction is a form of inference originally proposed by Aristotle, but significantly developed by Charles Peirce. It happens in a case context, when an anomaly is noticed in a stream of observations. Noticing something that stands out leads to one or more plausible explanations (Douven, 2017). Cases can raise a great number of abductive explanations, so there is a need to gauge their usefulness. Analogical reasoning can help. In analogical reasoning, a comparison is made between two domains that highlights similarities and supports a hypothesis pertaining to something as yet unobserved in the second domain, but observed in the first (Bartha 2013). Finally, phronetic reasoning can supply the grounds for applying knowledge to a new problem, especially when the ability to draw analytical inferences through ‘objective’ scientific reasoning is curtailed (Thomas, 2011). Again, we can look back to Aristotle, who introduced phronetic reasoning, a process that starts in the middle of already knowing, with the knower’s biases (prejudices) essentially opening up to what is next to be understood. Thomas (2011, p. 24), drawing on others, describes phronesis as involving a “combination of knowledge, judgement and taste, together producing a discernment, through which we see links, discover patterns, make generalisations, create explanatory propositions . . . all the time emerging out of our experience”. As elaborated by Gadamer (see Thomas, 2011, p. 21), another case creates in a person an ‘image’ that is tentative, but sufficient to be seen and heard in the context of another’s experience . . . but then used in the context of one’s own: it is essentially visual and dialogic. http://dx.doi.org/10.20377/cgn-48 Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 99 University of Bamberg Press Although travelling to the site of the case of interest is desirable, it can be impractical. Students, however, drawing on published documents, evaluations and academic case studies can extract a great deal of the basic story, be it about congestion-relief in Stockholm or species- preservation in France. Following Gadamer’s observation, the student–practitioner can see and hear another story from their own vantage point. That is, the exemplar enables a person to ‘see’ the prospective case by a process of iterating between what can be assessed directly and that which can only be anticipated or seen in the ‘mind’s eye’. As noted by Schön (1983), professionals have a capacity to undertake a “conversation” with a past exemplar, to carry a particular result of that conversation forward as a lesson, and then to apply reflections prospectively to the next assignment. If a learner approaches unique cases, suitably prepared by an initial ‘situation analysis’ and with a view to triggering plausible hypotheses through the process of abduction, resulting hypotheses can then be reflected on in light of their implications for action (via phronesis), rather than their fit with any posited pattern, such as a theory or diagnostic category (Thomas 2010). There is, of course, no certainty that the ‘right’ lessons will be learned, but this situation affects all learning. Learning Outcomes and Further Opportunities Many of the methods used in prospective case analysis can continue to be used after a change is implemented, by iterating a revised prospective case accounting for new observations and hypotheses. After the course, it is hoped that students will be inspired to undertake complexity- aware behaviours, such as monitoring for anomalies (Room, 2011) in newly changed policy ecosystems, and continuing to mentally form and test hypotheses (Sanderson, 2009; Eppel, Turner, & Wolf, 2011). With such an ‘experiment-minded’ perspective, grounded philosophically in pragmatism, a practitioner searches for what does not fit the expected pattern, followed by actions (including ‘thought experiments’) that take into account new perspectives arising from plausible explanations about what is occurring. Some indicators that students have learning about complexity emerge in discussion and through signs that there is greater sensitivity to complexity when approaching challenging policy issues, and perhaps somewhat less insistence on ‘science’-oriented information when making decisions looking ahead. After the final presentations in the Policy Workshop, students individually record tips for others on post-it notes. Ideas are volunteered that recognise complexity, ambiguity and uncertainty, as well as the unavoidable contextuality of policymaking. Sample tips include: • Dealing with ambiguity: sometimes a ‘best guess’ is the best you can do—use own insights and judgements to inform as well as any literature • Bear in mind: context, context, context • Learn from what is out there • Different types of knowledge (e.g., tacit, intuitive) are valid • Use evidence and judgement: sometimes all you will have is your ‘best guess’ In sum, I hope, but cannot be sure, that students complete these case exercises with their faith in evidence still intact, but also prepared to reject sharp dichotomies, to value more highly varieties of knowledge, and to welcome the emergence and evolution of their own understanding within the constraints presented by their practice environments. Initial support for this desired outcome is shown in an assessment, mentioned earlier, at the end of the Policy Methods course. Students are allocated one ‘theme’ from Table 1 or from Schmidt’s (1993) types of knowledge. http://dx.doi.org/10.20377/cgn-48 Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 100 University of Bamberg Press With reference to a policy issue that they have been studying individually, they are asked to describe a challenge posed by the allocated theme and to explain and illustrate why it is a challenge. Each student prepares a slide and a three-minute presentation to the class, thus ensuring students learn from a variety of applications. Policy Workshop cases are assessed through a set of exercises, of which those on assessing context and stakeholders are particularly conducive to applying their complexity learning. In addition, a new assignment has been introduced, called ‘Learning Record’. Amongst other lessons they record, I will be able to scrutinise these assignments for evidence of complexity learning. Reflecting on the courses more generally, it is clear that complexity can be approached from a variety of philosophical perspectives, including my pragmatism-influenced one. I am aware, however, that when I teach philosophical perspectives, one of the motivations I offer students is that in a public-sector career they are bound to encounter views from different perspectives. It is part of their professional development to learn the hallmarks and distinctions among them, to know their own, and to be open to information presented from another direction. If more time were available, I would choose to go deeper into some of the philosophical underpinnings that guide various approaches, and how these might inform a comparative prospective case exercise. I have wondered about problematising ‘evidence’ and introducing ‘complexity’—however softly—early in the two-course sequence. Would it not be wiser to start simply, for example, with textbook methods of analysis? Yet, very little in my practitioner–students’ world is simple. Largely for this reason, but also inherent in my understanding of complexity and the future-focus of policy work, I do not rely on prepared teaching cases or the ‘case-method’ as used in some business schools. In a small, diverse environment such as they find themselves, students already have refined abilities to work with complexity. They just don’t so label their practices. If I can pull back the screen, and enthuse them with the potential to gain more traction with some new concepts, the learning objectives are already well-launched. Moreover, in a small country context, students (as practitioners) are accustomed to looking elsewhere for policy evidence. Formalising the prospective case comparison, with a stream of complexity concepts running through it, appears to augment students’ capabilities without either losing them in a heavily theoretical treatment or taking them philosophically too far from their ‘day-job’ in evidence-handling. References Banks, G. (2009). Challenges of evidence-based policy-making. Australian Government Productivity Commission, speech delivered as the ANZSOG/ANU Public Lecture Series, Canberra, Feb 4. https://www.pc.gov.au/news-media/speeches/cs20090204/20090204- evidence-based-policy.pdf. Bartha, P. (2013). Analogy and analogical reasoning. In E. N. Zalta, (Ed.). The Stanford encyclopedia of philosophy (Fall ed.). http://plato.stanford.edu/archives/fall2013/entries/reasoning- analogy/. Barzelay, M. (2007). Learning from second-hand experience: Methodology for extrapolation- oriented case research. Governance, 20(3), 521–543. Börjesson, M., Eliasson, J., Hugosson, M., & Brundell-Freij, K. (2012). The Stockholm congestion charges—5 years on. Effects, acceptability and lessons learnt. Transport Policy, 20, 1–12. http://dx.doi.org/10.20377/cgn-48 http://plato.stanford.edu/archives/fall2013/entries/reasoning-analogy/ http://plato.stanford.edu/archives/fall2013/entries/reasoning-analogy/ Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 101 University of Bamberg Press Douven, I. (2017). Abduction. In E. N. Zalta (Ed.). The Stanford encyclopedia of philosophy (Summer ed.). http://plato.stanford.edu/archives/sum2017/entries/abduction/. Eppel, E. (2012). What does it take to make surprises less surprising? The contribution of complexity theory to anticipation in public management. Public Management Review, 14(7), 881–902. doi:10.1080/14719037.2011.650055. Eppel, E., Turner, D., & Wolf, A. (2011). Complex policy implementation: The role of experimentation and learning. In B. Ryan & D. Gill (Eds.). Future state: Directions for public management in New Zealand (pp. 182–212). Wellington: Victoria University Press. Head, B. W. (2008). Three lenses of evidence-based policy. The Australian Journal of Public Administration, 67(1), 1–11. doi: 10.1111/j.1467-8500.2007.00564.x. Majone, G. (1989). Evidence, argument and persuasion in the policy process. New Haven and London: Yale University Press. McKaughan, D. J. (2008). From ugly duckling to swan: C. S. Peirce, abduction, and the pursuit of scientific theories. Transactions of the Charles S. Peirce Society, 44(3), 446–468. Parsell, C., Fitzpatrick, S., & Busch-Geertsema, V. (2014). Common ground in Australia: An object lesson in evidence hierarchies and policy transfer. Housing Studies, 29(1), 69–87. Pawson, R. (2013). The science of evaluation: A realist manifesto. London: Sage. Prewitt, K., Schwandt, T. A., & Straf, M.L. (Eds.). (2012). Using science as evidence in public policy. Washington: National Academies Press. Room, G. (2011). Complexity, institutions and public policy: Agile decision-making in a turbulent world. Cheltenham, UK; Northampton, MA: Edward Elgar. Rose, R. (2002) Ten steps in learning lessons from abroad, EUI Working Papers 2002/5. http://cadmus.eui.eu/bitstream/handle/1814/1763/RSCAS_2002_05b.pdf?sequence=1. Rose, R. (2005). Learning from comparative public policy: A practical guide. London and New York: Routledge. Sanderson, I. (2009). Intelligent policy making for a complex world: Pragmatism, evidence and learning. Political Studies, 57, 699–719. doi:10.1111/j.1467-9248.2009.00791.x. Schmidt, M. R. (1993). Grout: Alternative kinds of knowledge and why they are ignored. Public Administration Review, 53(6), 525–530. Schön, D. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books. Smith, K., & Lawrence, G. (2014). Flooding and food security: A case study of community resilience in Rockhampton. Rural Society, 23(3), 216–228. doi: 10.1080/10371656.2014.11082066. Steyaert, P., Barzman, M., Billaud, J.-P., Brives, H., Hubert, B., Ollivier, G., & Roche, B. (2007). The role of knowledge and research in facilitating social learning among stakeholders in natural resources management in the French Atlantic coastal wetlands. Environmental Science and Policy, 10(6), 537–550. Thomas, G. (2010). Doing case study: Abduction not induction, phronesis not theory. Qualitative Inquiry, 16(7), 575–582. http://dx.doi.org/10.20377/cgn-48 Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, p. 92-102 DOI: http://dx.doi.org/10.20377/cgn-48 102 University of Bamberg Press Thomas, G. (2011). The case: Generalisation, theory and phronesis in case study. Oxford Review of Education, 37(1), 21–35. doi: 10.1080/03054985.2010.521622. Wolf, A., & Baehler, K. (2017). Learning transferable lessons from single cases in comparative policy analysis. Journal of Comparative Policy Analysis. doi: 10.1080/13876988.2017.1399578. http://dx.doi.org/10.20377/cgn-48