Microsoft Word - Peace Ed Assessment (IFP) updated proof.docx Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 93 Volume 13 Number 2 (2019): 93-116 ISSN 2578-6857 http://www.infactispax.org/journal Towards an Agreement on Learning Outcomes For Peace Education Moritz Bilagher Principal Evaluation Specialist, Internal Oversight Service, United Nations Educational, Scientific and Cultural Organization (UNESCO) The ideas and opinions expressed in this article are those of the author and do not necessarily represent the view of UNESCO. Abstract As both the value of educational assessment for educational development and peace education for social development are increasingly recognised, it is of concern that peace education has as of yet no widely accepted assessment methodology. This may be due to the absence of agreed learning objectives. Instead, peace education programmes tend to be evaluated as interventions to directly achieve peace, bypassing the need for learning outcomes. Using Delphi methodology, this study enquired how we could arrive at learning outcomes for peace education. This instance of Delphi was organised with a group of 16 experts in peace education. This Delphi found a difference between the social purpose of peace education and its learning outcomes. While its social purpose is peace, to be education, it must have learning objectives. While peace education is understood as education on group identity and diversity, this can be engaged with cognitively and non- cognitively, suggesting different types of outcomes. The Delphi concluded that learning outcomes in peace education culminate in inter-cultural communication skills, combining cognitive with non-cognitive characteristics. The offered understandings are underpinned by a relational conception of peace that is open- ended and non-utopian. Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 94 Keywords: peace education, inter-cultural communication skills, learning outcomes, learning objectives, educational assessment, educational evaluation A concern that evaluation in peace education is weak has existed since at least the beginning of this century (Ashton, 2007, cited in Amani Williams, 2015; Harris, 2003; Malhotra & Liyanage, 2005, cited in Lazarus, 2015; Nevo & Brem, 2002/2009; Salomon, 2004, cited in Duckworth et al., 2012). In their seminal review of peace education evaluations, covering the period 1981 to 2000, Nevo and Brem (2009) found 79 studies measuring the effectiveness of peace education programmes or projects. While most of these (80–90%) were found to have been successful, Nevo and Brem found shortcomings in the evaluation studies themselves, including a lack of delayed post-testing and attention to generalisability (p. 275). Harris (2003) agreed with Nevo and Brem that “there have been very few rigorous quantitative or qualitative evaluations of peace education efforts” (p. 8). Suggesting that not much changed in this regard, since January 2010 we find only six articles1 that, in one way or another, evaluate a peace education programme or project in the Journal of Peace Education. These articles themselves often lament the absence of evaluation in peace education (Duckworth et al., 2012; Kester, 2013). In response to these concerns, there has been a number of attempts to develop theoretical foundations to evaluate peace education programmes. First, Nevo and Brem (2009) themselves designed an ‘orientation map’ for peace education programmes to specify their ‘facets’ such as purpose; (targeted) age of participants; major didactic approach; duration; and, moving into the area of evaluation, research design and method of measurement (pp. 272-273). In reaction to Nevo and Brem, Salomon and Kuppermintz (2002) developed ‘mapping sentences’ of peace education evaluations considering: programme attributes; socio-political context; implementation criteria; domains of changes; targets; and the criteria for assessing change (p. 8). More recently, an ambitious project was undertaken to clarify concepts, methodologies and techniques in peace education evaluation, culminating in the work Peace Education Evaluation: Learning From Experience and Exploring Prospects (Del Felice et al., 2015). Evaluation and assessment 1 Apart from those cited in the remainder of this paragraph, they include: Akgun and Araz, 2014; Goldberg and Ron, 2014; and Trinder et al., 2010. Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 95 What is problematic in these efforts is that they (almost) invariably address evaluation of peace education rather than assessment of the achievement of peace education learning outcomes. The difference between these concepts is rooted in that formal education is almost entirely defined by a curriculum or the “(i) why; (ii) what; (iii) when; (iv) where; (v) how; (vi) and with whom to learn” (Braslavsky, undated, p. 1). Within this definition, the ‘what’ is represented by learning outcomes, i.e. the knowledge, skills and other characteristics learners are expected to achieve in a course, programme or module. Curricula thus address a learner. This suggests a logic different from that of many, specifically non-formal peace education programmes, which often focus on programme objectives rather than learners, using education and learners as means to an end that, in that logic, transcends her or him. In the programme logic, peace is more important than the learner. In the educational logic, learners are more important than the subject. The difference between the concepts of assessment and evaluation runs parallel to these two logics: the education one, in which the learner is an end in itself, as one is to a curriculum, is connected with assessment; the programme logic, which is usually concerned with objectives of collective development, is connected with evaluation. Indeed, in the context of education, evaluation is understood to denote the formulation of value judgements as to the achievement of programme objectives, while assessment is interested in what the participants in a programme have learned or, in the words of Nusche et al. (2012): The term ‘assessment’ is used to refer to judgements on individual pupil performance and achievement of learning goals. It covers classroom-based assessment as well as large-scale external tests and examinations. … the term ‘evaluation’ is used to refer to judgements on the effectiveness of schools, school systems and policies. (p. 24) Peace education professionals, however, often seem to use the terms evaluation and assessment interchangeably. For example, the mentioned volume by Del Felice et al. includes a contribution titled ‘Assessing Peace Education at the National Level’ (Barbeito Thonon & Ospina, 2015) in which the authors explicitly said they were not interested in addressing what participants had learnt (p. 239). This confusion between assessment and evaluation may be due to the fact that educational evaluation often uses assessment data to arrive at “judgements on the effectiveness of schools, school systems and policies” (Nusche et al., 2012, p. 24) or programmes. However, it may also be a strategy to conveniently overcome the circumstance that peace education is an umbrella concept covering a range of subjects, including “conflict resolution education, multicultural education, development education, world order studies (or international education), human Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 96 rights education and environmental education” (Hung, 2007, p. 40) or still other subjects (Bajaj, 2008; Duckworth, 2008; Harris & Morrison, 2013; Hung, 2007; and Salomon, 2009). After all, this heterogeneity signals the absence of a unified curriculum, from which broadly agreed learning outcomes may be derived. However this may be, when applying this distinction, it becomes clear that peace education evaluation, especially in non-formal education contexts, does not regularly use assessment data. To give one example, Obura (2002) evaluated the effectiveness of a peace education programme in two refugee camps in northern Kenya, Dadaab and Kakuma, on the extent to which they had become more peaceful after the programme, using a pre-test post-test design. The evaluation considered whether instances of human rights violations or violence had decreased, while an assessment of learning was not part of the evaluation design. The assumption seemed to be that to know the effectiveness of the programme it was unnecessary to assess what participants had learned. It was considered sufficient to know whether the programme had achieved its objective: more peaceful camps. (Incidentally, in formal contexts, where curricula do exist, assessment data may be used, both for high-stakes and informal purposes; however, such cases are often not reported in the academic literature. In addition, such curricula can widely vary among one another.) In line with the observation that evaluation may include assessment data, what characterises programme evaluation is mainly research design, for example pre-test post-test experimental designs. It can be implemented without knowing what has been learned. What characterises assessment, on the other hand, is mainly method, for example, tests (Robson, 2002, p. 292 ff.), essays and observations. This prevalence of design over method in work published in the field of peace education is reflected in its theory. For example, Nevo and Brem (2009) mentioned the facet of research design before the method of measurement; and their one traditional assessment tool, knowledge mastery tests, only as the sixth element within that facet, after official statistics, e.g. rates of mixed marriages (p. 273). In their discussion of what is missing in peace education evaluation, all facets except method were addressed. Research question There thus appears to be a gap in the literature on evaluation in peace education in that it mostly focuses on evaluation, rather than assessment. Yet this is understandable: in a sequential chain, the issue (a) that peace education evaluation tends to not consider assessment data seems to be rooted in that it (b) is problematic to do so as there exists no established methodology for assessment in peace Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 97 education. Developing such a methodology would (c) require agreed learning objectives to determine what to measure but (d) given the contested nature of peace education it seems challenging to agree on learning outcomes. The absence of learning outcomes in peace education thus seems to be the difficulty underlying absence of assessment in peace education evaluation. For this reason, it seems that for assessment in peace education to be possible, learning objectives will have to be agreed upon. This, in turn, might help clarify what unites peace education programmes per se. To address this issue, I will attempt to answer the question: what could the generic learning outcomes of peace education be? Methodology Delphi methodology I decided to address the above-mentioned questions using Delphi methodology, the consensus methodology with the most extensive track record of successful applications. Delphi, as Dalkey and Helmer (1963) argued, is appropriate to “obtain the most reliable consensus of opinion of a group of experts” (p. 1). In particular, as Rescher (1969) suggested, it offers a useful framework for addressing normative or value questions. In essence, it consists of a series of rounds of data collection (interviews or questionnaire administration), applied to a set number of experts, with feedback between rounds, with the aim to achieve a consensus on a specific issue. In this case, the issue is derived from the research question: what could the generic learning outcomes of peace education be? This process culminated in the expert consensus, which is reflected in a ‘statement of principles’ (Annex I). While Cuhls (undated) suggested a minimum of two rounds of Delphi (p. 93), Hsu and Sandford (2007) argued that, according to most authors, three rounds are sufficient, in particular if literature is available on the subject (p. 2). As this study targeted experts with limited time, three rounds were applied as follows: (1) administration of an open-ended questionnaire to identify main issues. This could be administered as a semi-structured interview. (2) Elaboration of a preliminary position paper (1-2 pp.), i.e. the statement of principles, based on questionnaire data. In this phase, participants could provide feedback. (3) Elaboration of final position paper, accounting for the feedback received in Round 2. In this phase, participants could indicate their adherence, if applicable, and identify points of dissent. It served as a phase of validation of the statement in terms of capturing the expert consensus. Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 98 Research participants Selection of participants is critical in Delphi (Hsu and Sandford, 2007), both in terms of quantity and selection criteria, even though “no exact criterion currently listed in the literature concerning the selection of Delphi participants” (p. 3) exists. On the basis of Schulz & Renn, 2009, I looked for experts in the field of peace education representing several positions in respect of the subject to be addressed (p. 14) and, to do so, considered four variables as proxies for different views: geographic spread, to avoid cultural bias; gender, to avoid gender bias; institutional affiliation; and area of focus in peace education. I identified the experts through Internet-based and other searches. In all cases, they were at least one and often both of the following: (a) authors of significant articles in leading peer reviewed Journals and / or book chapters; and / or (b) medium to high-profile officials or researchers affiliated with organisations working in the field of peace education. In terms of quantity of participants, Hsu and Sandford (2007) suggested that 10-15 constitute a reasonable number. Similarly, Schulz and Renn (2009) recommended that a group not exceed 16-25 persons (p. 14). While 22 persons agreed to participate in the study initially, after several reminders, 16 responses were received for Round 1 of the Delphi. This was a good group size, considering the above-mentioned indications. In line with Schulz and Renn (2009), it could be argued that too large a group of experts would not only become unmanageable for the precise, in-depth analyses that consensus methodologies require, but also that it would make consensus more difficult to elicit. So then, ultimately six women and ten men participated in this Delphi: ten from universities; three from UN-related organisations; one from a think-tank; one from an NGO; and one from a religious organisation. In terms of geographical location, five participants were from the United States, two each from Costa Rica and Germany; one from Colombia, Republic of Korea, United Kingdom, Qatar, Democratic Republic of Congo, Nigeria and Austria each. Methods and analysis The questionnaire – or interview protocol – was structured in such a way that Question 1 asked for the biography of the participant in relation to peace education; Question 2 addressed the concept of peace; Question 3 addressed the concept of peace education; and Question 4, learning outcomes in peace education. The concept of educational assessment in general was addressed in Question 5, while Questions 6 and 7 focused on assessment in the context of peace education. Question 8, finally, asked whether participants would like to add anything. Thus Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 99 participants could comment on the questionnaire if they thought something was missing. The questionnaire contained only open-ended questions. In terms of analytical strategy, given that I was after a theory on assessment in peace education, grounded in the empirical data of the study, my requirement seemed filled by Glaser and Strauss’ grounded theory approach (1967/2009). This is based on the two main techniques of coding and categorisation of data, roughly as follows (cf. Robson, 2002, p. 493): (1) first review of data – i.e. questionnaires and interview transcripts - or open coding, to identify key ideas (conceptual categories); (2) second review of data or axial coding to find relationships between these categories and identify key themes; (3) third review of data (selective coding), to account for these relationships through core categories underpinning the themes. Research ethics All participants enjoyed anonymity. While there are few specific known ethical issues related to Delphi, there is one worth highlighting: while I generally attributed ideas that were not mine, to their owners, the statement of principles reflects a consensus developed through a process that I facilitated, while the ideas emanated from this process itself. For example, one participant mentioned that peace is the social purpose, not a learning outcome of peace education. This idea made its way to the statement, due to its explanatory value for the theory but, while I agree with it, it was not my original idea. However, I could not attribute it to a specific participant, due to the anonymity clause. The added value of my contribution then lay in bringing ideas together, choosing one over the other and crafting a coherent set of principles. The analysis of the data was mine and the statement shared, reflecting a consensus. Data analysis Understandings of peace In the analysis of the respondents’ views of peace, Galtung’s (1975) definitions of positive and negative peace proved a useful device. Roughly, three types of response were received to the question what peace is: firstly, those that emphasised the importance of negative peace, absence of violence; secondly, those that presented some kind of positive definition of peace; and thirdly, those that indicated that peace cannot be defined, should not be defined or did not define it for another reason. In this last category we encountered a respondent who, for pragmatic reasons, did not define peace (P10); one who argued that there is not one peace, but Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 100 that there are many “peaces” (P16); and one who suggested that the concept of peace is best approximated by the action of peace-building, i.e. “a collective effort toward co-constructing possibility” (P8). Yet, the same participant sympathised with a definition of peace as described in Article 16.f of the Earth Charter (The Earth Charter Initiative, 2000): Recognize that peace is the wholeness created by right relationships with oneself, other persons, other cultures, other life, Earth, and the larger whole of which all are a part. Among the other two groups, consisting of those participants that emphasised the importance of negative peace and those that presented some type of positive definition of peace, there was an extent of overlap: of the five respondents that mentioned the importance of negative peace – i.e. peace as absence of overt violence - for understanding peace (P3, P4, P6, P7, P9), only one said that all of peace is covered by negative peace. All but one either supplemented the importance of negative peace with a positive definition (P3, P7) or mentioned the importance of negative peace in conjunction with positive peace. When peace was defined positively, it was, again, defined in either of two ways: as a ‘state’ (of integrated being, P2; of justice, P7; of safety, P11; or of harmony, P12), or as a ‘process’ rather than a desired end state. In the case of one answer (P7), these two ways co-existed, although the concept of peace-as-process prevailed: I perceive the state of being peaceful to be the absence of both direct and structural violence and oppression, or in a more positive term, the state of having justice, equity, non-violence, healthy ecology, human rights, diversity, pluralistic democracy and care for oneself and others ensured and secured in a society. While the definition as such may be organized around its “state of being” as an end product, in approaching to “peace” in my own work, I treat “peace” more as a process than the state of being to be arrived at. In cases where peace was defined as a state, this was described in divergent ways, for example as in the preceding quotation, or as “presence of trust, respect, equality and expanding social justice and inclusion” (P1), rendering its meaningful understanding challenging. At first sight, it seemed difficult to find a common denominator to such a wide range of ‘things’ peace was thought of as. Yet, with few if any exceptions, all Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 101 things that peace were considered to be, were relational – describing an organisation of a relation between self and others, self and nature and even self and self. For example, harmony was harmony in a relation between self and others, or also self and self; justice, a principle organising a relation between self and others; safety referring to a freedom from vulnerability or risk, again in a relation; trust, respect and equality all existing only in relations. Without an ‘object’, a ‘subject’ cannot trust or respect – there has to be an object, even if that is oneself. The counterpart for peace that was least controversial was, explicitly or implicitly, groups, such as women, children and refugees (P5). The identification of group relations as object of peace connected well with Salomon’s (2009) view from the literature that peace in the context of peace education addresses conflict between groups, rather than inter-personal conflict (see also Harris, 2008, p. 249). While in the only case where groups were mentioned explicitly, individuals were, too (P12), the numerous references to social justice, structural violence and human rights (i.a. P1, P2, P7, P9) pointed to peace in terms of relations between groups. This notion that peace is, in the first place, relational, was supplemented by positive definitions of it as ‘open-ended’ or a ‘process’ rather than a state. It was striking that one participant (P13) labelled this as non-utopian: This [Earth Charter] definition also connotes peace as a very active, ongoing process – rather than a utopian end state and also hints to the fact that pursuit of peace is multidimensional, comprehensive, and holistic. The implications of this are significant. If peace were utopian and if, as per the historicist perspective, it were the necessary outcome of human history, then, as Popper (1994) said, everything that stands between the now and peace, e.g. capitalism in the opinion of Marx, could legitimately be eliminated (p. 50). This creates a conundrum, in as far as ‘elimination’ is not an activity usually associated with peace. While the conception that peace be understood as a process would overcome this dilemma, it was not directly clear how this could be imagined. Fortunately, one participant (P3) gave a clue to solve this, referring to Galtung (1967) who had said that positive peace is made of ‘positive relations’, of which one is justice, which he saw as synonymous with equality (p. 15). This helped clarify how to perceive positive peace as process: equality, understood as a main form of justice, is not (only) a state, but (also) a process. It is something that can be enacted in concrete instances on a more or less continuous basis. So, peace as process are the things that are done to further the equality of human beings and Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 102 groups. This, for peace, is axiomatic: ‘right relations’ are relations of fundamental equality. Understandings of peace education At first sight, the views of the experts of what peace education is (or should be), mirrored the issue described in the Introduction: that it is ‘something’ – a process, project, or programme – that should bring ‘something else’ about, whereby the first something (the independent variable, IV, or factor) X was not further defined, but the second thing (the dependent variable DV, or outcome) Y was peace. In understanding peace, Galtung’s definitions could be used. This, of course, related exactly to the initial problem underpinning this study: that, while there might have been consensus on what peace education should ultimately do, there was still no consensus on what its identity, its core, that is to say, its curriculum should be. Thus, peace education itself remained somewhat of a black box. Consequently, peace education was often defined as a process to bring about peace, without specifying what this process should consist of. For example, one participant (P1) argued that: “Peace education embraces activities and curriculum that make our schools more peaceful within, and empowers them to contribute to peace in their global and local communities”. This definition said what peace education was supposed to do, but not how it was supposed to do this. This appeared as the fundamental challenge for peace education: it was considered ‘education for peace’, but what kind of education this is, was not clear. To give an example of this (P11): “Peace education is a process through which we can educate our children, or the community on ways to live in harmony with the people around us, and the environment around us”. There were, however, three variations to this theme that helped understand the concept of peace education to a greater extent. The first of these took an intermediate step between education and its intended outcomes. For example (P7): Education for peace is a kind of education (or any education) that encourages and facilitates the learners to develop their life-long commitments to the values of, and actions for, peace (as defined above) and their enduring capacities to materialize such commitments. So here we had the final aims of education – “peace (as defined above)” – but also the outcome of equipping learners with such skills, dispositions and abilities as to bring peace about, i.e. the development of their “commitments” to values and their “capacities” (abilities, skills) to give substance to these commitments. This was, in Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 103 a way, a step beyond the issue of peace education into that of peace education learning outcomes and even accompanied by a ‘mini-taxonomy’, where such objectives were thought of as having one affective, or value-related component (commitments) and one skills-related one. In addition, a relation was laid out between these, where one was configured as the enabler of the other. This shift was fundamental: in line with the problematique discussed in the Introduction, the emphasis was not anymore on the wider environment, or even the school, but on learners; thus, we could move from a ‘programme logic’ to an educational one. In summary, this three-step configuration, of X (education) leading to Y (peace) by means of Z (given learning outcomes) was a step forward on the IV (independent variable) → DV (dependent variable) or ‘an X [whatever X means] leading to Y’-logic in that it moved the focus to learners. A similar pattern appeared more than once in responses, for example (P12): “Peace education is [X] the process of acquiring [Z] knowledge, skills and dispositions that empowers individuals, groups and institutions to [Y] resolve conflicts harmoniously and fairly”; or (P13): “Peace education [X] prepares and nurtures learners with [Z] the knowledge, skills, capacities and attitudes necessary to [Y] critically understand, confront and end violence, war and injustice and promote a culture of peace.” Only very few experts gave substance to the ‘X’: peace education. For example, and even though technically speaking learners were not mentioned (P4): “peace education is [X] teaching and learning processes (including formal and non- formal) and practices that foster [Z] knowledge, attitudes and behaviours that support the realization of [Y] peace” (emphasis mine). This additional step was crucial to understanding the essence of peace education in a number of ways. The ‘thin’ XYZ-configuration (where peace education was mentioned, but not defined), as in the previous paragraphs, established the primacy of the learner before the subject, that is to say, it clarified that peace education is about learners before being about peace. This sets it apart from, for instance, peace studies, of which the primary focus is on peace. The ‘thick’ XYZ-configuration, on the other hand, where peace education is defined as teaching and learning processes, went one step further: in line with the notion that education is not just or even mainly about outcomes but also process, this suggested that it is, in one way or another, an engagement. This led to a demarcation line between what is an educational experience for peace and what is any other ‘peace-related’ experience: it has to be an engagement of a learner. What we would classify as peace education would be experiences such as those stipulated in Allport’s (1979) chapter on contact, in his The Nature of Prejudice (Ch. 30), later called ‘Contact Hypothesis’ or ‘Contact Theory’ and of which social travel is an Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 104 example.2 According to this theory, if certain conditions are in place, contact with members of another group could lead to greater appreciation of that group. If we accept the common denominator of peace education as ‘education for peace’, which seems to emerge from the above analysis, in combination with the definition of peace as relational, and mainly addressing inter-group issues, it would seem that lines of demarcation of which subjects fall under the umbrella of peace education may be drawn. For example, education for sustainable development would not be a manifestation of peace education as it does not necessarily deal with differences between groups. In contrast, the following seem to belong to it (Table 1): Type Main focus Inter-cultural education Knowledge of (other) cultures International education Knowledge of (other) countries Sexuality education Knowledge of (other) gender(s) Religious education3 Knowledge of (other) religions Understandings of learning outcomes in peace education So far we found that peace, in the context of peace education, is relational and open- ended and peaceful relations are, in essence, relations of equality, especially of groups; and that peace education is a type of engagement with a learner in respect of peace, which gives primacy to the learner, and where the focus is on group diversity and identities. These views and, in particular, the notions that, in peace education, learners come before peace and thus peace education’s main focus is not on peace but on a skills-set for the learner as peace-maker, were re-confirmed in the discussion on learning outcomes in peace education. For example, a respondent (P2) argued that: “Outcomes should focus on the development of capacities— capacities of critical thinking, inquiry, reflection, etc. as well as the development of a sense of justice.” This view was possibly most clearly formulated by P13: 2 Allport (1979) described some further examples of this under the name of ‘intercultural education’ (p. 264). 3 These are relatively generic indications. For example, in religious education, Maudarbux (2016) created a distinction between religious education and inter-religious education (p. 461). Tab. 1: Types of peace education Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 105 I think it’s important to distinguish between social purposes of education and learning outcomes. Social purposes are those conditions in society that education seeks to maintain, change or transform. The social purposes of peace education – nurturing a culture of peace – are rather open ended. Learning outcomes, on the other hand, I see as more specific and rooted in the development of fundamental capacities or competencies seen as necessary for personal, social and political engagement. While programme evaluation in peace education assesses achievement of the social purpose of peace education, which is peace (in practice often negative peace), only educational assessment can help evaluate achievement of its learning outcomes. This illustrates why the gap identified in the Introduction is real: if peace education programmes are evaluated but learners not assessed, the achievement of the social purpose rather than its learning outcomes (changes in learners) will be considered the purpose of peace education. When addressing the question what the learning outcomes of peace education should be, two initial findings emerged: first, that learning outcomes should be relational, i.e. should reflect issues of identity and diversity. Second, that the sheer diversity of types of learning objectives (knowledge, skills, dispositions, attitudes, etc.) suggested that one category were insufficient to capture all peace education learning outcomes. To illustrate the first of these points, several respondents emphasised the importance of involvement with diversity, with ‘Others’, e.g. (P1): “In terms of longer term outcomes and sustainability, I’d look for a student’s organically (… self motivated) continued engagement with including diverse ‘Others’ and involvement in related activities”. This position was echoed by others, for example (P4): There may be core outcomes of peace education related to skills that are promoted through the teaching and learning processes, including: knowledge specific to the context in which peace education is taking place; analytical skills; and attitudes and skills that support positive relationships with others and acceptance of differences. This relationality, as a learning outcome, was captured by P3 in one word: empathy. This suggested realisation of One-ness: empathy may be interpreted as the understanding of the Other from an awareness that, somehow, one is the other. In P3’s words, interpreted, the capacity of empathy establishes the link with the Other cognitively and affectively: cognitively, in that it enables one to see from the perspective of someone else and affectively in the sense that this perception moves Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 106 the subject in one way or another, that is to say, involves him or her emotionally. This empathy can occur between others as individuals but, importantly, it can also be directed at others as a group. Another type of outcome mentioned was knowledge of identities and differences and conflicts these can give rise to. For example, one respondent (P1) proposed: “being able to describe what a peaceable community would look like, articulate some causes of violence and name means accessible to the student of interrupting that violence (dialogue, community problem solving workshops, advocacy).” Interestingly, this was mentioned in the context of a scale implying a difference between lower and higher order skills in the cognitive domain, suggesting that knowledge precedes advanced skills. Another example was one according to which the outcomes of peace education include, presumably in this order (P4): “knowledge specific to the context in which peace education is taking place; analytical skills; and attitudes and skills that support positive relationships with others and acceptance of differences.” Although these conceptions seemed unclear at first sight, I slowly saw an order emerging. Using the concept of relationality as an organising principle I saw, in line with Heidegger’s (1927/2006) philosophy outlined in Being and Time, that Sorge (care, or concern) was a starting point for understanding peace education learning outcomes. What is needed to engage with identity and difference is an engagement-based care. I call this: recognition. This is a pre-requisite and, in sequential terms, a first stage were difference is recognised4 (which refers, first, to one’s own identity and, secondly, to diversity, i.e. others’ identities). This diversity may occur on a number of axes such as ethnicity; gender; religion; etc. One taxonomy applicable here is Krathwohl et al.’s (1964) affective domain, elaborated based on the concept of ‘responsiveness’. This seems similar to Heidegger’s Sorge. Once basic recognition of identity and diversity, through Sorge, responsiveness or engagement would have occurred, knowledge of these recognised identities and diversities can be generated. This would follow P1’s order where “being able to describe what a peaceable community would look like”, or recognition, would be followed by the (cognitive) skill to “articulate some causes of violence”. This largely coincides with what Allport (1979) called an ‘informational approach’ to reducing prejudice between groups (p. 485). It represents a stage where information is gathered on difference, i.e. first one’s identity and, secondly, the other’s, constituting diversity. This may involve several 4 Like the word realisation, used above, the word recognition is ambiguous: similarly to acknowledgment it means at once factual acknowledgement, which is cognitive, and appreciation, which belongs to the affective realm. Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 107 educational subjects, including history (addressing migration, war, colonisation and in general the narrative of events that lead to group formation), geography (borders, geographical distribution of cultures) and religious studies. In terms of taxonomy of learning outcomes, here, Bloom et al.’s Book I would apply. As knowledge seems necessary, but not sufficient in peace education, in line with P1, after gathering knowledge, one would expect “continued engagement with including diverse ‘Others’” be it in terms of gender, culture or other identifiers. This area has been theorised by Allport, in his The Nature of Prejudice. On this third stage, learning would be experiential, involving social travel or a similar type of educational event, or the intercultural and interreligious encounter as described by Gill (2016, p. 490). Incidentally, this order of stages, where knowledge precedes the stage of contact, was implied by Allport (1979) who argued that for contact to lead to reduction of prejudice, prior education is a pre-requisite (pp. 264-265). Without this, contact may consolidate rather than diminish prejudice. P4’s view, quoted above that, in terms of learning objectives, peace education should flow into “attitudes and skills that support positive relationships with others and acceptance of differences” suggested that there should exist a learning objective above ‘mere’ contact. We saw that she referred to both ‘attitudes’ and ‘skills’. Similarly, P2 argued that: “Outcomes should focus on the development of capacities—capacities of critical thinking, inquiry, reflection, etc. as well as the development of a sense of justice.” These views suggested that, following from a recognition of identity and diversity; a stage of gathering knowledge about these identities; and one of engagement with diversity; a stage should follow where relations that were recognised, explored and engaged with, become ‘right relations’, i.e. relations of fundamental equality. Such relations seemed to be addressed by Kohlberg’s (1981) stage of moral development called principles of individual conscience. But what kind of outcomes may apply here? I assert that the answer to this question is hidden in P4’s view, which referred to attitudes and skills that support positive relationships with others and acceptance of differences. There exists a specific skills set uniting such attitudes and skills: inter-cultural communication skills, roughly as described by Lustig and Koester (2006). This is the ‘know how’ allowing for relational peace: communication between (individuals from) different groups to bridge identity and diversity, self and others. Communication, moreover, is a process; it promotes and expresses understanding at the same time as it is both expressing oneself and listening, which Gill (2016) calls “a compassionate act” (p. 492) and requires traits such as empathy (p. 456). Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 108 The hierarchy presented in this section might be visualised as follows: Stage Domain Learning outcome ↓ Recognition: of identity and difference Affective Responsiveness Knowledge: of own and other groups Cognitive Lower order (memorisation), higher order (analysis) Contact: with other groups Conative Open-ended [hopefully reduced prejudice] Moral (conscience) Multiple Inter-cultural communication Towards expert consensus While Round 1 of the Delphi led to the statement of principles, and Round 2 elicited feedback on this, in Round 3 the 16 participants were asked whether they agreed, grosso modo, with the statement; agreed with qualifications; or did not agree. Responses were received from nine participants and, of these, eight agreed ‘grosso modo’ and one ‘with reservations’. None said that they did not agree. This suggests that the statement reflects, to a reasonable extent, expert consensus on learning outcomes in peace education. Discussion and conclusion Thus the answer to the question ‘what could the generic learning outcomes of peace education be?’ is: learning outcomes in peace education should be imagined on different stages, from an (affective) recognition of identity and difference; via (cognitive) knowledge of own and other groups; and (conative) contact with other groups, culminating in inter-cultural communication skills, which combines all of the mentioned domains. Of course, further research may (in-) validate this model. For example, if these are really sequential stages, it is not clear whether they are related to specific age groups or educational levels. While the idea that contact without knowledge can reinforce prejudice is found in the literature (Allport, 1979, pp. 264-265), it seems problematic to argue that contact between individuals belonging to different groups (e.g., gender groups) should not occur until a certain amount of knowledge is in place. I believe that there are two other main consequences of my findings for academic debate and research. First, this study validated the relevance of the problematique underpinning the research question. This suggests that the peace Tab. 2: Taxonomy of learning outcomes in peace education Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 109 education research community should engage more with educational assessment. Finally, the study found that several subjects that tended to be classified as peace education, such as human rights education or environmental education (Hung, 2007, p. 40), are not as per the consensus. While surprising it is convenient as there seems to exist a significant conceptual lack of clarity in peace education, which hinders the development of assessment theory and practice. In this sense, I hope that my enquiry into learning outcomes for peace education may contribute to a greater understanding of this field itself. References Akgun, S. & Araz, A. (2014). The effects of conflict resolution education on conflict resolution skills, social competence, and aggression in Turkish elementary school students. Journal of Peace Education, Vol. 11, Nr. 1, 30- 45. Allport, G.W. (1979). The nature of prejudice. Cambridge, MA: Perseus Books. (Originally published 1954) Amani Williams, H.M. (2015). Peaceableness as raison d’être, process and evaluation. In C. Del Felice, A. Karako & A. Wisler (Eds.), Peace education evaluation: Learning from experience and exploring prospects (3-18). Charlotte, NC: Information Age Publishing. Ashton, C.V. (2007). Using theory of change to enhance Peace Education evaluation. Conflict Resolution Quarterly, Vol. 25, Nr. 1, 39-53. Barbeito Thonon, C. and Ospina, J. (2015). Assessing peace education at the national level: Challenges and possibilities. In C. Del Felice, A. Karako & A. Wisler (Eds.), Peace education evaluation: Learning from experience and exploring prospects (239-253). Charlotte, NC: Information Age Publishing. Bloom, B.S. [and 33 others] (1984). Taxonomy of educational objectives: Book 1 cognitive domain. New York, NY: Longman. (Originally published 1956) Braslavsky, C. (undated). The curriculum. Retrieved 10 September 2017 from: http://www.unhas.ac.id/hasbi/LKPP/Hasbi-KBK-SOFTSKILL- UNISTAFF-SCL/Hasbi-UNISTAFF- Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 110 DOCUMEN/MODUL%20UNISTAFF%20SURABAYA%202006/QTL/c urriculum%20development/cecilia%20e.pdf Cuhls, K. (undated). Delphi method. Retrieved 21 September 2014 from: http://www.unido.org/fileadmin/import/16959_DelphiMethod.pdf Dalkey, N. & Helmer, O. (1963). An experimental application of the Delphi Method to use of experts. Management Science, Vol. 9, 458–467. Del Felice, C., Karako, A. & Wisler, A. (2015). Introduction. In C. Del Felice, A. Karako & A. Wisler (Eds.), Peace education evaluation: Learning from experience and exploring prospects (xv-xxii). Charlotte, NC: Information Age Publishing. Duckworth, C.L., Allen, B. & Triguba Williams, T. (2012). What do students learn when we teach peace? A qualitative assessment of a theater peace program, Journal of Peace Education, Vol. 9, Nr. 1, 81-99. Galtung, J. (1967). Theories of peace: A synthetic approach to peace thinking. Oslo: International Peace Research Institute. Galtung, J. (1975). Peace: Research, education, action – Essays in peace research, I. Copenhagen: Christian Ejlers. Gill, S. (2016). Universities as spaces for engaging the other: A pedagogy of encounter for intercultural and interreligious education. International Review of Education, Vol. 62, 483–500. Glaser, B.G. & Strauss, A.L. (2009). The discovery of grounded theory: Strategies for qualitative research. New Brunswick, NJ: Aldine Transaction. (Originally published 1967) Goldberg, T. & Ron, Y. (2014). ‘Look, each side says something different’: The impact of competing history teaching approaches on Jewish and Arab adolescents’ discussions of the Jewish–Arab conflict. Journal of Peace Education, Vol. 11, Nr. 1, 1-29. Harris, I. (2003). Peace education evaluation. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, April 21-25. Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 111 Harris, I.M. (2008). The promises and pitfalls of peace education evaluation. In J. Lin, E. Brantmeier & C. Bruhn (Eds.), Transforming education for peace (245-264). Charlotte, NC: Information Age Publishing. Heidegger, M. (2006). Sein und Zeit [Being and time]. Tübingen: Max Niemeyer Verlag. (Originally published 1927) Hsu, C.C. & Sandford, B.A. (2007). The Delphi technique: Making sense of consensus Practical Assessment, Research and Evaluation. Vol. 12, Nr. 10, 1–8. Hung, R. (2007). Is ecological sustainability consonant or dissonant with human rights? Identifying theoretical issues in peace education, Journal of Peace Education, Vol. 4, Nr. 1, 39–55. Kester, K. (2013). Peace education: An impact assessment of a case study of UNESCO-APCEIU and the University for Peace. Journal of Peace Education, Vol. 10, Nr. 2, 157-171. Kohlberg, L. (1981). The meaning and measurement of moral development. Volume XIII. Heinz Werner lecture series. Worcester, MA: Clarke University Press. Krathwohl, D.R., Bloom, B.S. & Masia, B.B. (1964). Taxonomy of educational objectives: Book 2 affective domain. New York, NY: Longman. Lazarus, N. (2015). Evaluating seeds of peace: Assessing long-term impact in volatile contexts. In C. Del Felice, A. Karako & A. Wisler. Peace education evaluation: Learning from experience and exploring prospects (3-18). Charlotte, NC: Information Age Publishing. Lustig, M.W. & Koester, J. (2006). Intercultural competence: Interpersonal communication across cultures. Boston, MA: Pearson. Malhotra, D. & Liyanage, S. (2005). Long-term effects of peace workshops in protracted conflicts. Journal of Conflict Resolution, Vol. 49, Nr. 6, 908-924. Maudarbux, M.B. (2016). Interreligious education: Conceptualising a needs assessment framework for curriculum design in plural societies (with special reference to Mauritius). International Review of Education, Vol. 62, 459–481. Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 112 Nevo, B. & Brem, I. (2009). Peace education programs and the evaluation of their effectiveness. In G. Salomon & B. Nevo (Eds.), Peace education: The concept, principles, and practices around the world (271-282). New York, NY: Psychology Press, 271-282. (Originally published 2002) Nusche, D., Laveault, D., MacBeath, J. & Santiago, P. (2012). OECD Reviews of evaluation and assessment in education: New Zealand 2011. Paris: OECD Publishing. Obura, A.P. (2002). Peace Education programme in Dadaab and Kakuma, Kenya: Evaluation summary. New York: UNHCR. Popper, K. (1994). La lezione di questo secolo [The lesson of this century]. Venezia: Marsilio. Rescher, N. (1969). Delphi and values. Santa Monica, CA: RAND Corporation. Robson, C. (2002). Real world research. Oxford: Blackwell. Salomon, G. & Kuppermintz, H. (2002). The evaluation of peace education programs: Main considerations and criteria. Haifa: The Center for Research on Peace Education, University of Haifa. Salomon, G. (2004). Does peace education make a difference in the context of an intractable conflict? Peace and Conflict: Journal of Peace Psychology, Vol. 10, Nr. 3, 257–274. Salomon, G. (2009). The nature of peace education: Not all programs are created equal. In G. Salomon & B. Nevo. Peace education: The concept, principles, and practices around the world (3-13). New York, NY: Psychology Press. (Originally published 2002) Schulz, M. & Renn, O. (2009). Das Gruppendelphi: Konzept und Fragebogenkonstruktion [The group-based Delphi: Concept and questionnaire development]. Wiesbaden: VS Verlag für Sozialwissenschaften. The Earth Charter Initiative (2000). The Earth Charter. Retrieved 19 March 2017 from: http://earthcharter.org/invent/images/uploads/echarter_english.pdf Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 113 Trinder, M., Wertheim, E.H., Freeman, E., Sanson, A., Richardson, S. & Hunt, S. (2010). Comparison of the effectiveness of two forms of the Enhancing Relationships in School Communities project for promoting cooperative conflict resolution education in Australian primary schools. Journal of Peace Education, Vol. 7, Nr. 1, 85-105. Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 114 Assessment of the achievement of learning outcomes in peace education Statement of principles 1. This statement formulates principles for the assessment of learning outcomes in peace education. 2. The assessment of learning outcomes in peace education is important because it helps to: (a) assess learner progress, thus creating learning opportunities for both learners and teachers; (b) assess effectiveness of the education initiative and its pedagogy to achieve its intended outcomes; and (c) for research purposes. The question that is critical in educational assessment per se, i.e. assessment of whom, for whom and with what purpose is equally relevant to assessment in peace education. 3. The domain that these principles cover consists of four main concepts: peace; education (for peace); learning outcomes (of peace education); and assessment (of the learning outcomes of peace education), which relate to each other in the given sequential order. To describe how the achievement of learning outcomes in peace education can be assessed, these concepts are clarified below. 4. Peace is the central concept of peace education. While the absence of violence (‘negative peace’) is critical to peace, its durability can only be guaranteed by establishing ‘positive peace’. Rather than a utopian end-state, this embodies the Kantian ideas that (a) every human being, as a subject of Reason, is an end in itself and (b) that one should act in such a way that one’s action could be a universal law. The postulate underpinning these ideas is that of human beings’ fundamental equivalence. 5. Peace is thus a type of relation between self and others, or identities. As the greatest threat to human equivalence is the idea that one (group) identity is of greater worth than another, in peace education peace is, primarily, inter- group peace. It pursues the ideal of unity - in essence and value - in diversity of appearance. 6. The concept of education is similar to that of peace, in that it is non-utopian – that is to say, it does not lead to an end state or, in the words of Dewey: “Education is not preparation for life, it is life itself”. In addition, for both Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 115 peace and education the human mind is central, which is where the bulwarks of peace are constructed, as per UNESCO’s Constitution, and it is the focus of attention of education. 7. While there exists learning without teaching, education without learning is unthinkable. Peace education initiatives, therefore, by definition include learner engagement. 8. Thus, while peace education may operate as an umbrella term, its criteria of demarcation are that it denotes (a) learner engagement (b) in relation to identity - self and other(s) – focusing, primarily, on inter-group peace. Groups can be defined in a number of ways, including gender, national and religious. Thus peace education includes international education, religious education and sex education. Global citizenship education (GCED) may be considered archetypal peace education. 9. As peace education focuses on identity, and on the human mind’s engagement with this, its learning outcomes should be human-centred, i.e. focus on the learner’s development rather than peace itself, although this is its ultimate objective. Peace is the social purpose of peace education, not its learning outcome. 10. The above suggests four levels of engagement, as per § 7 above, that may in principle take place on any educational level, with associated learning outcomes: a. The recognition of self and difference, that is to say, of identity and identities, which are likely to be multiple, overlapping and possibly conflicting; b. Knowledge of self and others, that is to say, knowledge of these identities (gender, national, religious and socio-economic); c. Engagement with others, that is to say, contact with carriers of other identities, either physical or otherwise; d. Identification with principles of universal justice (as per § 4 and 5 above), and as reflected in the UDHR, that is to say, on the assumption that communication is key to identity and identification, and that identity and Otherness are essentially cultural, the acquisition of intercultural communication skills. 11. These outcomes stand in an iterative relation with both inner peace, i.e. peace with one’s self or one’s identity, and peace with nature, given the Volume 13 Number 2 (2019): 93-116 http://www.infactispax.org/journal 116 symbiotic relation between human being and nature, and the extent to which this defines our identities. 12. These levels of learning outcomes are associated with types of assessment, as follows: a. Assessment on this level seems irrelevant to objectives § 2 a and b; b. Traditional methods for cognitive assessment; c. Verification of whether the contact has taken place, accompanied by a debriefing in the form of a dialogue or group discussion, without evaluation; d. Foreign language skills; critical reading and writing; presentation and debating. 13. As an integral part of peace education, its assessment should be ethical. As the extent to which one succeeds in peace education is the extent to which others benefit, assessment in peace education should treat competition with caution. 14. Psychometrics can be used in assessment in peace education. It is recommendable to develop a context, input, process and product (CIPP-) model for assessment in peace education, given the importance of context and collective in relation to the individual and their achievement in this subject. 15. Similarly to how peace education has the potential to transform education, so assessment in peace education has the potential to transform assessment overall, that is to say, to help reflect on how educational assessment can help achieve peace.