North B. (2016.) Developing CEFR illustrative descriptors of aspects of mediation. International Online Journal of Education and Teaching (IOJET), 3(2).132-140. http://iojet.org/index.php/IOJET/article/view/125/131 Received: 25.02.2016 Received in revised form: 04.04.2016 Accepted: 07.04.2016 DEVELOPING CEFR ILLUSTRATIVE DESCRIPTORS OF ASPECTS OF MEDIATION Brian North EUROCENTRES bjnorth@eurocentres.com Brian North has spent his career at the interface between research, practical implementation of innovation in language teaching, and quality management. As a result he combines developmental, investigative and statistical skills with management and coordination expertise. He has extensively collaborated with the Council of Europe as a project coordinator and consultant. He spent most of his long career at Eurocentres, an official NGO to the Council of Europe since 1968, which teaches languages where they are spoken. There he developed an integrated approach to curriculum, assessment and certification and implemented it across languages. In his PhD thesis he developed the levels and descriptors for the CEFR. After coordinating the 1991 intergovernmental Symposium that recommended the CEFR, he co-authored the CEFR itself, the prototype European Language Portfolio, the Manual for relating examinations to the CEFR, Eaquals’ Core Inventories for English and French, and the European Profiling Grid for language teacher competences (EPG). He was Chair of Eaquals from 2005 to 2010. His most recent publications are The CEFR in Practice, CUP, 2014 and “Putting the Common European Framework of Reference to good use,” Language Teaching, 2014. Currently he is coordinating the Council of Europe project to extend the CEFR descriptors. Copyright by Informascope. Material published and so copyrighted may not be published elsewhere without. http://iojet.org/index.php/IOJET/article/view/125/131 mailto:bjnorth@eurocentres.com 132 DEVELOPING CEFR ILLUSTRATIVE DESCRIPTORS OF ASPECTS OF MEDIATION Brian North bjnorth@eurocentres.com Abstract This article reports on a project commissioned and coordinated by the Council of Europe to develop descriptors for the category ‘Mediation’ in the Common European Framework of Reference for Languages. Mediation is the fourth communicative language activity presented in CEFR Chapter 4, complementing reception, interaction and production. Descriptors for mediation had not been developed in the 1993–6 Swiss National Research Project that produced the original set of illustrative descriptors for the CEFR. The work took place in the context of a wider 2013–6 project to provide an extended set of CEFR illustrative descriptors. The article describes the way in which the approach taken to mediation in the project is broader than the one taken in the presentation of mediation in the CEFR text in 2001. In addition to information transfer (conveying received information) the new scheme also embraces the construction of meaning and relational mediation: the process of establishing and managing interpersonal relationships in order to create a positive, collaborative environment. Descriptors were also developed for other, related, areas. The article briefly summarises the three phases of validation to which the draft descriptors were subjected before being calibrated to the mathematic scale underlying the CEFR’s levels and descriptors. 1. The Context of the Project The context to the development of descriptors for mediation for the Common European Framework of Reference for Languages: learning, teaching, assessment (CEFR: Council of Europe 2001) is a dual initiative on the part of the Council of Europe decided at a meeting in Strasbourg in May 2013. Firstly, a text was commissioned to situate the CEFR’s descriptive scheme within the broader educational context for language learning that has developed over the past 20 years, and in particular on the role of mediation in schools (Coste and Cavalli 2015). Secondly, the current writer was asked to put together a team to update the CEFR’s illustrative descriptors, including adding new scales for mediation, for online learning and for reactions to literature. The original idea was to select and adapt to the CEFR house style descriptors from CEFR-related projects that had scientifically calibrated their descriptors to the CEFR levels, as the original illustrative descriptors had been so calibrated. Unfortunately, no validated descriptors for mediation existed, as they did for updating the descriptors for reception, interaction and production, and such descriptors that did exist confined themselves to an information-transfer view of mediation. These sources were useful for developing scales for conveying received information, but for other categories of mediation, descriptors needed to be developed from scratch. 2. Mediation in the CEFR The CEFR pioneered the introduction of mediation in language teaching and learning to describe a fourth category for communicative language activities in addition to reception, interaction and production. “In both the receptive and productive modes, the written and/or oral activities of mediation make communication possible between persons who are unable, for mailto:bjnorth@eurocentres.com 133 whatever reason to communicate with each other directly. Translation or interpretation, a paraphrase, summary or record, provides for a third party a (re)formulation of a source text to which this third party does not have direct access. Mediation language activities, (re)processing an existing text, occupy an important place in the normal linguistic functioning of our societies.” (Council of Europe 2001: 14) Mediation as a language activity is probably more difficult than reception, interaction or production because it usually involves all of them, together with a cognitive and interpersonal challenge. This is because mediation always involves a self-effacing bridging effort to ‘get something across,’ to facilitate the (mutual) understanding of other people. Coste and Cavalli (2015:12) take this idea a stage further and claim: “In all cases, the aim of the mediation process, defined in the most general terms, is to reduce the gap between two poles that are distant from or in tension with each other.” It was indeed Daniel Coste who introduced mediation into the CEFR, to replace North’s (1992) fourth proposed category for communicative language activities in the series reception, interaction, production, processing. But in its treatment of mediation, the CEFR maintained this focus on “processing” in the sense of summarising and/or explaining to another person the content of a text to which they do not have access, generally because of linguistic, cultural, semantic or technical barriers. An emphasis on mediation as cross- linguistic information transfer has also been maintained in the descriptors produced for Profile Deutsch (Glaboniat et al 2005), in cross-linguistic test tasks developed in Greece (Stathopoulou 2013), in a new, plurilingual Matura oral examination offered in Austria (Piribauer et al 2014). 3. A Broader View of Mediation However, the concept of mediation has a very long history of much broader interpretation, originating in the role of an intermediary in diplomacy and conflict resolution, that has today developed into wider forms of conflict avoidance and counselling services. The relevance of mediation to the educational domain derives from Vgotsky’s (1978) theories, in which it is seen as a core feature when adults, siblings and peers interact with a child. In the resulting socio-constructivist / social cultural view of learning (Lantolf 2000) mediation has recently been developed into the concept of ‘languaging’ in order to mediate meaning, a process that takes two forms: collaborative dialogue and private speech (Swain, Kinnear & Steinman, 2015: 32). The latter concerns the individual mediating meaning for themselves and is often internal and invisible. It is the former with which we are primarily concerned in this article. Collaborative dialogue in fact already appears in the CEFR as the scale for Cooperating under interaction strategies. In addition, the emphasis in the CEFR on the mediator as an intermediary between different interlocutors underlines its social, collaborative vision of language (Piccardo 2012). A fundamental point about mediation is that it is not concerned with the linguistic expression of a speaker. Instead, the focus is on the role of language in processes like creating the space and conditions for communication and/or learning, constructing new meaning, encouraging others to construct or understand new meaning, passing on information in an appropriate form, and simplifying, elaborating, illustrating or otherwise adapting input in order to facilitate these processes (mediation strategies). Mediation always involves bridging across spaces, facilitating understanding. The context can be social (e.g. Wall & Dunne 2012), pedagogic (e.g. Mercer & Hodgkinson 2008), cultural (e.g. Zarate et al 2004), linguistic (e.g. Statholopoulou 2015) or in the workplace (e.g. Lüdi 2014). 134 4. Categories for Descriptor Scales Coste & Cavalli’s (2015) propose a distinction between cognitive mediation, the process of facilitating access to knowledge and concepts, particularly when an individual may be unable to access this directly on his /her own, and relational mediation, the process of establishing and managing interpersonal relationships, usually in order to improve the conditions for cognitive mediation. After experimentation with the categories social, pedagogic, cultural and linguistic mediation that were mentioned above, this distinction between cognitive and relational mediation was adopted as the basis for the mediation categories. The vast majority of the descriptors were inspired by articles from the fields of pedagogic, cultural and social mediation, plus cross-linguistic studies. The procedure followed was to collect relevant behaviours mentioned in sources, edit them into descriptors for a particular scale, analyse what content strands were represented in that scale, and then draft further descriptors for other (usually lower) levels. This initial collection of descriptors was presented to consultants in a meeting held in Strasbourg in June 2014. As a result of their feedback, the collection was revised before the first of nine meetings of an ad hoc Mediation Group 1 . The descriptors were then extensively edited between October 2014 and January 2015 before being subjected to the three phases of validation activities described in the next section. The result of the entire process was the set of descriptor scales shown in Table 1. Table 1 MEDIATION ACTIVITIES Relational Mediation (spoken) - Establishing a positive atmosphere - Creating pluricultural space Added after Phase 2 - Facilitating collaborative interaction with peers - Managing interaction in plenary and in groups - Resolving delicate situations and disputes Cognitive Mediation Constructing meaning (spoken) - Collaborating to construct meaning - Generating conceptual talk Conveying received meaning (spoken) - Relaying specific information - Explaining data (e.g. in graphs, diagrams, charts etc.) - Processing - Interpreting - Spoken translation of written text (Sight translation) Added after Phase 1 Conveying received meaning (written) - Relaying specific information - Explaining data (e.g. in graphs, diagrams, charts etc.) - Processing - Translating MEDIATION STRATEGIES - Linking to previous knowledge - Amplifying text 1 Brian North, Coreen Docherty, Tim Goodier, Hanan Khalifa, Ángeles Ortega, Enrica Piccardo, Maria Stathalpoulou, Sauli Takala 135 - Streamlining text Restructuring text (in appropriate discourse culture) Dropped after Phase 1 - Breaking down complicated information Added after Phase 1 - Visually representing information - Adjusting language OTHER NEW SCALES CREATED - Online conversation and discussion - Goal-oriented online transactions and collaboration - Expressing a personal response to literature and art - Analysis and criticism of literature and art - Exploiting pluricultural repertoire in intercultural encounters - Plurilingual comprehension Added after Phase 3 - Exploiting plurilingual repertoire EXISTING SCALES – EXTENSIVELY REVISED - Receptive strategies: identifying cues and inferring Added after Phase 3 - Phonological Control - Sound recognition and articulation - Prosodic features (stress, rhythm and intonation) Added after Phase 2 5. The Validation Between February and November 2015 these descriptors were then subjected to a validation process organised in three phases:  Phase 1: allocating descriptors to categories  Phase 2: assigning descriptors to CEFR levels  Phase 3: rating a person’s ability to perform what is described by a descriptor Phase 1: In these face-to-face, distance workshops, 472 draft descriptors were presented on a series of 30 overlapping sets, with sets being allocated to different institutes. 137 institutes took part, giving approximately 990 respondents. The task was to identify the intended category of about 60 descriptors presented in random order, to rate them for clarity, pedagogical usefulness and relation to real world language use, and to suggest improvements to the wording. The instructions for the workshop itself were as follows: Step 1: Tick in one column only (i.e. for a category, or for “Can’t Decide” or “Drop this descriptor”), referring to the Descriptor Sheet. Pay attention to the item numbers!! If you tick “Drop this descriptor,” proceed to the next descriptor. Do not do Step 2 in this case. If you tick “Can’t decide” you may still do Step 2 if you wish. Step 2: Judge the quality of the descriptor. Tick Y/N (=Yes or No) in the column for each of the three criteria accordingly. Step 3: (Optional): If you want to suggest changes to a descriptor, write these directly on the Descriptor Sheet, put your name on the sheet and return it to the coordinator. If the pair could not agree, they were also to mark “Can’t decide.” A completed response sheet looked like Image 1: 136 Image 1: Completed response sheet The appearance of the Descriptor Sheets was considerably more varied. Some people provided, as requested, considered proposals for amendments – and these proved very useful indeed. Others underlined, circled, and scribbled and wrote comments in a manner that required some interpretation. Electronic data collection, as used in Phases 2 and 3, resulted in clearer feedback. . The results from the all sheets like that shown in Image 1 were collated so as to list together all the results for the same descriptor, as shown below for descriptor 230. As can be seen, this item, for Linking to previous knowledge (LINK) was overwhelmingly allocated to the correct category. To analyse the data, coefficients based on percentages were developed, following Eichelmann (2015). For example, descriptors were expected to score 70% on the ratings for clarity, pedagogical usefulness and relation to real world language use, and any descriptor which 16% of the respondents wanted to drop was dropped. ID Set Scale Scale comparisons* CAN'T DECIDE DROP IT GEN STIM PROsp LINK RESTR AMPL STREA ADJUS INFO 230 11 LINK 20 44 230 16 LINK 1 9 2 11 230 17 LINK 13 1 2 22 230 20 LINK 12 1 1 33 137 Phase 2: In a second series of workshops, following activities familiarising participants with the CEFR levels, the respondents were asked to judge the CEFR level of 426 descriptors, presented in a series of 23 overlapping questionnaires. 189 institutions from 45 countries and 1294 persons took part. This is fairly remarkable considering that this phase took place in May-June, which is an extremely busy time of year for educational and examination institutes. The aim was that each form should be rated by 40–50 persons so that (given the overlapping) each descriptor should be rated by 100 persons. This aim was met for all descriptor scales: the lowest number of respondents being 151 and the highest 273 for a new scale written after Workshop 1 Breaking down complicated information. The task in Phase 2 was the following. For each descriptor, respondents were asked the question: At what CEFR level do you think a person can do what is defined in the descriptor? They were asked to choose between all the 10 bands of proficiency associated with the CEFR common reference levels: Pre A1 (called ‘Tourist’ in the CEFR), A1, A2, A2+, B1, B1+, B2, B2+, C1 and C2. They made their decisions first on paper and then, after reflection and perhaps discussion, entered them into their computer. The original intention of Workshop 2 had been just to sort the descriptors into levels, following the task of sorting into categories in Workshop 1. Calibration would then follow with an online survey in the autumn. However, because of the large numbers of respondents for all scales, (minimum 151 responses) it was decided to also do Rasch analysis of the ratings of descriptor level in the workshop, and thus calibrate at least some of them before the autumn. The ‘difficulty values’ of 20 calibrated CEFR descriptors that had been included in the survey were used to “anchor” the scale from this new analysis to the scale underlying the CEFR levels (North 2000). The results enabled the working group to identify and either eliminate or rewrite descriptors which were assigned to a wide spread of levels. 192 descriptors for which there was a very high consensus of agreement on level were calibrated in this phase. Phase 3: The calibration of the remaining descriptors, plus new descriptors written for lower levels, then took place in an online survey carried out in English and French. Because, after Phase 2, 192 descriptors for mediation now had difficulty values on the scale underlying the CEFR levels, there was a good pool from which to select anchor items for all the new categories in order to link back to the CEFR levels mathematically. This meant that each of the new categories now had its own anchor items, independently of the original CEFR descriptors. Since some of the categories were a considerable way away from the reception, interaction and production activities that had been calibrated in the original research (North 2000), this was a considerable advantage. 365 descriptors (including 12 CEFR anchor items and 62 anchor items selected from the 90 that had been very stably calibrated in Phase 2) were once again presented on a series of 23 overlapping questionnaires. Distribution to the institutions that had taken part in Phase 2 was supplemented with an open call through organisations such as FIPLV (International federation of language teachers), the ECML (European Centre for Modern Languages), Eaquals (Evaluation and Accreditation of Quality in Language Services), EALTA (European Association for Language Testing and Assessment) and the CASLT/ ACPLS (Canadian Association of Second Language Teachers). As a result there were a total of 3503 usable responses. 138 The Phase 3 survey task replicated the way the vast majority of the original CEFR illustrative descriptors had been calibrated. The instruction was to think about how a person whom respondents knew very well (this could be themselves or someone else) would perform in a second/foreign language and answer the following question: Could you, or the person concerned, do what is described in the descriptor? The same 0–4 rating scale used in the original CEFR descriptor research (North 2000) was used in answering the question. After the analysis had been completed, there were now a total of 395 validated descriptors that had been calibrated in Phases 2 and 3. 6. Follow-ups Two follow up surveys were also carried out in February 2016 on plurlilingual / pluricultural competences and on phonology respectively. The plurilingual/pluricultural survey was undertaken to expand the number of descriptors for this area. After consulting experts on plurilingualism, descriptors for Plurilingual comprehension were put into their own separate scale. Sources on pluralistic approaches were also consulted and used to expand the scale Exploiting pluricultural repertoire. This follow-up survey used the Phase 2 methodology. Phonology was investigated for two reasons. Firstly, after the work on plurilingual competences, the existing CEFR scale for Phonological control appeared outdated, comparing performance to a native speaker rather than stating what the user/learner could actually do. Secondly, recent work on phonology had emphasized the centrality of intelligibility and the importance of providing points of reference for teachers in relation to sound articulation and prosodic features. The survey repeated the Phase 3 methodology, with respondents assessing the overall phonological control, the sound articulation and the use of prosodic features of user/learners shown on videos created by the CIEP in 2008, using 27 descriptors to rate three performances. At the end, respondents were asked to allocate the descriptors – plus another seven for sound recognition – to CEFR levels (the Phase 2 methodology). 7. Conclusion One of the surprising aspects of the project was the overall stability of the concepts scaled at the different CEFR levels. The coherence between aspects of proficiency scaled in the new descriptors with those in the descriptors from the original research (North 2000) was remarkable. This was a considerable achievement considering that the areas now being described were very different, the type of informants was substantially different, from 45 countries rather than just from Switzerland, and the not inconsiderable fact that the research took place 20 years later. The proposed wider set of CEFR illustrative descriptors, including those from the mediation project, will be presented to users in an extended consultative process before publication, following the precedent set with the CEFR itself. The process will start in June 2016 with a consultative meeting with experts. Then, the descriptors will be circulated in autumn 2016 in a preliminary edition for a period of wider consultation and piloting until later 2017. 139 References Coste, D. and Cavalli, M. (2015). Education, mobility, otherness: The mediation functions of schools. Strasbourg: Council of Europe DGII – Directorate General of Democracy, Language Policy Unit. Council of Europe (2001). Common European framework of reference for languages: learning, teaching, assessment. Cambridge: Cambridge University Press. Eichelmann (2015). Der gemeinsame europäische Referenzrahmen für Sprachen und die Hochschulspezifik. am Beispiel UNIcert. Unpublished PhD thesis, University of Dresden. Glaboniat, M., Müller, M., Rusch, P., Schmitz, H. and Wertenschlag, L. (2005). Profile deutsch A1 - C2. Lernzielbestimmungen, Kannbeschreibungen, Kommunikative Mittel. München: Langenscheidt. Lantolf, J. P. (2000). Sociocultural theory and second language learning. Oxford: Oxford University Press. Lüdi, G. (2014). Dynamics and management of linguistic diversity in companies and institutes of higher educations: Results from the DYLAN project. In P. Gromes and A. Hu (Eds.), Plurilingual education: Policies – practices – language development (pp113– 138). Hamburg: Hamburg Studies on Linguistic Diversity 3, John Benjamins. Mercer N. and Hodgkinson (Eds.) (2008). Exploring talk in schools. London: Sage. North, B. (2000). The development of a common framework scale of language proficiency. New York: Peter Lang. North, B. (1992). European language portfolio: Some options for a working approach to design scales for proficiency. In: Council of Europe, Transparency and coherence in language learning in Europe: Objectives, assessment and certification. Symposium held in Rüschlikon, 10–16 November 1991 (pp158–174). Strasbourg: Council for Cultural Co- operation. Reprinted in Schärer R. & North B. Towards a Common European Framework for Reporting Language Competency, Washington D.C.:NFLC Occasional Paper, National Foreign Language Center. Piccardo, E. (2012). Médiation et apprentissage des langues: Pourquoi est-il temps de réfléchir à cette notion ? In ÉLA (Études de Linguistique Appliquée), 167, 285–297. Piribauer, G., Steinhuber, B, Atzlesberger, U., Mittendorfer, F., Ladstätter, T. Greinix,I, and Renner, H. (2014). Wegweiser: Mehrsprachigkeitsprüfungen. Linguistic mediation: “How to assess plurilingual oral competences within the final Austrian baccalaureate exam. Work in progress. Dokument für Brian North Council of Europe expert & coordinator on behalf of the Council of Europe, March 2014. CEBS: Center für Berufsbezogene Sprachen, Vienna. Stathopoulou, M. (2013). The linguistic characteristics of KPG written mediation tasks across levels. In N. Lavidas, T. Alexiou and A-M Sougari, (Eds.), Major trends in theoretical 140 and applied linguistics: Selected papers from the 20th ISTAL. London: Versita de Gruyter. Stathopoulou, M. (2015). Cross-language mediation in foreign language teaching and testing. Cleveland: Multilingual Matters. Swain, M., Kinnear, P. and Steinman, L. (2015). Sociocultural theory in second language education: An introduction through narratives (2 nd edn). Bristol: Multilingual Matters. Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press. Wall, J. A. and Dunne, T. C. (2012). Mediation research: A current review, Negotiation Journal, April 2012, 217–244. Zarate, G., A. Gohard-Radenkovic, A. and Lussier, D. (2004). Cultural mediation in language learning and teaching, Graz: European Centre for Modern Languages. Cook, V. J. (2007). The goals of ELT: Reproducing native-speakers or promoting multi- competence among second language users? In J. Cummins & C. Davison (Eds.), Handbook on English language teaching (pp. 237–248). Dordrecht, the Netherlands: Kluwer. Canagarajah, S. (2006). The place of world Englishes in composition: Pluralization continued. College Composition and Communication, 57, 586–619.