319 Studies in Second Language Learning and Teaching Department of English Studies, Faculty of Pedagogy and Fine Arts, Adam Mickiewicz University, Kalisz SSLLT 6 (2). 2016. 319-337 doi: 10.14746/ssllt.2016.6.2.7 http://www.ssllt.amu.edu.pl The effects of CLIL on mathematical content learning: A longitudinal study Jill Surmont Vrije Universiteit Brussel, Belgium jisurmon@vub.ac.be Esli Struys Vrije Universiteit Brussel, Belgium estruys@vub.ac.be Maurits Van Den Noort Vrije Universiteit Brussel, Belgium info@mauritsvandennoort.com Piet Van De Craen Vrije Universiteit Brussel, Belgium pvdcraen@vub.ac.be Abstract Previous research has shown that content and language integrated learning (CLIL), an educational approach that offers content courses through more than one educational language, increases metalinguistic awareness. This im- proved insight into language structures is supposed to extend beyond the lin- guistic domain. In the present study, the question whether pupils who learn in a CLIL environment outperform their traditionally schooled peers in math- ematics is investigated. In total, 107 pupils entered the study. All participants were in the first year of secondary education at a school in Ostend, in Flan- ders, the Dutch-speaking part of Belgium. Thirty-five pupils followed CLIL ed- ucation in a foreign language (French) and 72 followed traditional education that was given in the native language (Dutch). All participants were tested us- ing a mathematical test at the beginning of the year, after three months, and Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 320 after ten months. The first measurement of the mathematical scores showed that the two groups did not differ. In accordance with our hypothesis, the CLIL group scored higher than the non-CLIL group after ten months. Surprisingly, an effect was also found after three months. To conclude, CLIL appears to have a positive impact on the mathematical performance of pupils even after a short period of time. Keywords: content and language integrated learning; second language; teach- ing; math performance; longitudinal study 1. Introduction Content and language integrated learning (CLIL) refers to a teaching approach where subjects, or parts of subjects, are taught in a foreign language with dual- focused aims, namely the learning of content and the simultaneous learning of a foreign language (Marsh, 1994). CLIL combines a number of pedagogies that have proven to work well and, as a result, it has become a well-established part of education systems across Europe (e.g., Banegas, 2012; Bonnet, 2012; Nikula, Dalton-Puffer, & García, 2013; Pérez-Cañado, 2012). There are several reasons why CLIL is different from traditional language learning approaches. 1.1. Differences between CLIL and traditional language education First of all, CLIL is dual-focused and has both content and language goals, which creates a continuum with both goals at one end, without any specification of importance of one over the other (Coyle, 2006). The target language is used as a vehicle for communication and is therefore not the ultimate goal. In other words, CLIL is a task-based language learning approach by which language is learned through usage of the language while discussing content matter. This cre- ates a meaningful learning environment in which pupils have an immediate need to use the target language. Feedback is given through scaffolding, ensuring the pupils are constantly in what Vygotsky (1978) called “the zone of proximal development.” This means that the pupil is able to fulfill tasks because the teacher is helping, whereas alone the pupil would not be able to do so. A second difference from traditional language learning approaches is the way CLIL is pupil-centered rather than teacher-centered (González & Barbero, 2013). In many Western societies, the dominant teaching model is based on what Freire (1972) calls the “banking model.” This means that the teacher has the role of the expert and deposits information and skills into the memory bank The effects of CLIL on mathematical content learning: A longitudinal study 321 of the pupil. This banking model creates a teacher-controlled and teacher-led learning process. CLIL, on the other hand, has a more social-constructivist ap- proach, meaning that the emphasis is on the student experience and on the en- couragement of active student learning (Cummins, 2005; Marsh, 2012). The fo- cus of CLIL is therefore on interactive, mediated, and student-led learning, which requires social interaction between teachers and students. The third difference focuses on the resemblance of the language learning process in CLIL to the natural cognitive skill acquisition. There are two types of processes that interact during a learning process, namely implicit learning and explicit learning (Sun, Mathews, & Lane, 2007). Surmont, Van de Craen, Struys, and Somers (2014) compared this interaction with the way a child learns how to tie his/her laces: He/she needs to practise on his/her own, but when a parent or sibling provides explicit information on what to do with the laces, the child will figure out much faster how it is done. Research by Bialystok and Barac (2012) showed that when second languages are learned implicitly, the brain re- sponds the same way as it does when using the mother tongue. This does not imply that it is better, but it proves that the learning context has a major influ- ence on how something is processed. This can, for instance, explain previous neurolinguistic results in which bilingual brains were found to be more efficient than monolingual brains (Mondt, Struys, Balériaux, & Van de Craen, 2008). In traditional education, there are various reasons why implicit (language) learning is not used often enough. Sun, Mathews, and Lane (2007) described it as follows: “Most educational settings focus on teaching conceptual (explicit) knowledge rather than setting up an opportunity for gaining substantial experi- ential (mostly implicit) knowledge” (p. 1). In traditional language education, the emphasis is all too often on grammatical correctness and often teachers are un- able to give their students sufficient time to use and to practise using the target language due to time constraints. In CLIL education, on the other hand, students learn the language in a much more implicit way, as they have to use it to under- stand and communicate about the content of the course. They basically learn the language “along the way” (Surmont et al., 2014). Due to a much higher im- plicit learning aspect in CLIL classes, children benefit more from the interaction between implicit and explicit education. It is of course impossible to solely offer implicit or explicit learning. Rather, learning should be seen as a continuum in which both sides can never be fully tuned out (Sun et al., 2007), thus implying that traditional language courses are not obsolete in CLIL programs, because they provide the necessary, more explicit focus on language. Francis (2004) even suggests that it is not bilingualism per se that creates increased metalinguistic awareness (see further) but that the learning context in which the pupils are situ- ated has the largest influence. The combination of a CLIL class and a traditional Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 322 language class creates a learning environment in which the brain thrives, an en- vironment where explicit and implicit learning can take place (Ellis, 2008). A final difference between CLIL and traditional foreign language classes is the possible increased development of metalinguistic awareness. Metalinguistic awareness can be described as the ability to “reflect on and manipulate the structural features of languages” (Nagy & Anderson, 1998, p. 155). It permits reasoning and application of logic with language (Ter Kuile, Veldhuis, Van Veen, & Wicherts, 2011). Usage of metalinguistic awareness usually requires a mo- mentary shift of attention from content to form (Edwards & Kirkpatrick, 1999). In normal language use, metalinguistic awareness is hardly relevant as convey- ing the message is normally considered as more important than the linguistic structures used to convey it (Nagy & Anderson, 1995). Speakers do rely on it when they have to access deeper linguistic knowledge when, for example, cor- recting certain slips of the tongue (Fromkin, 1980) or deciphering an atypical form of speech or dialect (Kemper & Vernooy, 1993), when making puns or word jokes (Horgan, 1981), when linguistic ambiguities have to be resolved, when grammaticality or appropriateness has to be judged, or when decisions have to be made on deeper meanings or intentions based on word choices or paralin- guistic cues (Edwards & Kirkpatrick, 1999). 1.2. Increased metalinguistic awareness It is clear that the ability to reflect on language structures requires a higher level of abstract thinking and understanding. Research reports that metalinguistic skills (Bialystok, Peets, & Moreno, 2014; Hermanto, Moreno, & Bialystok, 2012) are an important factor that distinguishes bilingual from monolingual language users and learners (Bialystok, 2001; Jessner, 2006). It has been found that bilin- gual children have an increased metalinguistic awareness compared to mono- lingual children (Ransdell, Barbier, & Niit, 2006; Whitehurst & Lonigan, 1998). The influence of an increased metalinguistic awareness may extend the linguistic domain. For instance, studies have shown that bilingual pupils have an advantage in mathematics when they are highly competent in both languages, compared to their monolingual peers (Clarkson, 1992, 2007; Dawe, 1983). A possible explanation for this is the increased metalinguistic awareness of high ability bilingual students (Clarkson, 2007). Those bilinguals can compare the lan- guage structures of the languages they speak and come to a higher understand- ing of how each language structure functions on its own, whereas monolinguals do not (or rarely) question the way their language system is built up. These met- alinguistic abilities seem to be important factors in mathematical performance (e.g., the better metalinguistic skills of bilingual students allow them, for instance, The effects of CLIL on mathematical content learning: A longitudinal study 323 to self-correct when solving problems; moreover, bilingual students are more confident in their approach to solving difficult problems compared to monolin- gual students) (Clarkson, 2007). It can be argued that mathematics is a “language” (Arianrhod, 2005; Gold- haber, 2006), because it has its own set of rules, and with a finite number of sym- bols an infinite number of utterances can be created. Moreover, as in other lan- guages, students need to master this language in order to read, understand, write down, and discuss (mathematical) ideas (Thompson & Rubenstein, 2000). Orr (1987) even claimed that most difficulties that students have with mathematics are rooted in language. Orr noticed that the structure of the so called Black Eng- lish, that is, English spoken by many black students, causes their problems in mathematics. In her work, she illustrates the concepts which these students do not understand and which underlie mathematics, namely prepositions, conjunc- tions, and relative pronouns. If we accept that mathematics is a language (Usiskin, 1996), then it might be possible that bilinguals may acquire the language of math- ematics faster than their monolingual peers and may have a better insight into it. When it comes to CLIL, it is likely that pupils will benefit from an increased metalinguistic awareness, as the extra language provided in a meaningful con- text will create a deeper insight into language structures. Ter Kuile and col- leagues (2011) have shown that pupils in CLIL programs have a better insight into language structures, even with respect to an unknown language. When looking at the maths performance of CLIL pupils compared to their traditionally schooled peers, research indicates that CLIL pupils have an advantage (e.g., Mur- ray, 2010; Valladoloid, 1991; Van de Craen, Ceuleers, & Mondt, 2007). This not only confirms that CLIL has a positive influence on content knowledge but also that it stimulates cognitive development. The influence it seems to have on maths performance could be an indication that CLIL stimulates metalinguistic skills in such a way that the understanding of the language of maths (and/or science, as seen in Jäppinen, 2005) increases. 1.3. Research question and hypotheses The question in the present study is if monolingual pupils whose mathematics is delivered via CLIL outperform their traditionally schooled peers whose math- ematics is taught in their mother tongue. The first hypothesis of this study is that pupils who learn in a CLIL environment are likely to outperform their tradi- tionally schooled peers on mathematical tests, because CLIL is assumed to cre- ate an increased metalinguistic awareness (Mehisto & Morsh, 2011, p. 36) and, as a result, their understanding of the “language of math” (e.g., Arianrhod, 2005; Goldhaber, 2006) increases (Jäppinen, 2005). However, as the learning Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 324 process cannot be rushed, and a small change in the learning environment is unlikely to show its effect on a cognitive level immediately (Goswami, 2008), the second hypothesis is that differences between the CLIL group and the non-CLIL group will not be apparent after only a short period of time but will require a longer period of study, after which they will become visible. 2. Method 2.1. Participants The participants of this study were 107 pupils from the first year of secondary education in a school in Ostend, in Flanders, the Dutch-speaking part of Belgium. Thirty-five pupils had chosen to follow the CLIL course, while 72 had preferred the traditional curriculum. For ethical and legal reasons, it was not possible to randomly divide the 107 pupils in a CLIL group and a non-CLIL group; however, this selection procedure might have resulted in the so-called “creaming effect” (Bredenbröker, 2002, p. 146; Rumlich, 2013, p. 185) because it cannot be excluded that the two groups are different with respect to factors such as motivation, pa- rental support, and so forth (for a detailed discussion, see Jäkel, 2015). The CLIL course lasted one hour per week and the pupils of the CLIL group followed the CLIL course throughout the whole school year (i.e., 10 months). The pupils were taught in French with dual-focused aims, namely the learning of the subject (mathematics; content) and the simultaneous learning of the language (French; foreign language). The CLIL teaching was pupil-centered and interactive, there was a significant role for implicit learning, and the teacher helped the pupils in order to succeed. Aside from the fact that the CLIL group received their maths instruction through French, there were no significant differences in the teaching strategies or the curriculum content between the two groups, and all other in- struction was delivered in their native Dutch. Furthermore, there were no sig- nificant differences in the teaching strategies used for all other subjects of the curriculum between the CLIL and the non-CLIL pupils. All participants were being raised in Dutch only, and, although they all had had two years of introduction to French in primary education, their French proficiency level was so low that the secondary French teachers informed the researchers that they had to start from the very beginning. For this reason, the pupils in this research are considered to be at a very early stage of second language acquisition. Participants from both groups did not differ (p > .05) in age in years (the CLIL group: M = 12.23, SD = .39; the non-CLIL group: M = 12.36, SD = .45). Gender distributions were equal for both groups (Pearson Chi square = .47): In the CLIL group, there were 19 females and 16 males; in the non-CLIL group there were 34 females and 38 The effects of CLIL on mathematical content learning: A longitudinal study 325 males. Lastly, pupils in both groups had the same socio-economic status (SES) as measured by the educational attainment of both parents (Pearson Chi square for mother = .31; Pearson Chi square for father = .13). 2.2. Materials Three versions of a new mathematical test called the Mathematical Assessment Test-Help (MATH) have been developed by the researchers. At the time, there were no official standardized tests in Flanders (Belgium) to test mathematical knowledge in secondary education. Each of the three versions of the MATH con- sisted of the same kind of mathematical exercises and were presented in the native language of the pupils (Dutch). The test consisted of four different parts: I. Mental calculation, II. Numeracy, III. Mathematical applications and insight, IV. Geometry, measurement and arithmetic calculation; and it included both theo- retical as well as practical exercises covering arithmetic and geometry (for the three versions of the mathematical test, see the appendices of the online supporting information at http://ssllt.amu.edu.pl/download/docs/SSLLT%206(2)%20Surmont, %20Struys,%20Van%20Den%20Noort,%20Van%20De%20Craen%20Appendices.pdf). The correction of all tests was conducted by one corrector who was not familiar with the aims of the study, and the maximum score on the MATH was 100. 2.3. Procedure At the beginning of the school year, all pupils completed the first of the mathematical tests based on the official standards that have to be met in order to start secondary education. At that point, no CLIL education had been introduced to the pupils (here- after we refer to this time point as T = 0 throughout the paper). After three months, all pupils completed a second mathematical test (T = 1), and after 10 months all pu- pils completed a final mathematical test (T = 2). The study was approved and con- ducted according to the ethical guidelines of the Vrije Universiteit Brussel, Belgium and the Declaration of Helsinki (World Medical Organization, 1996). 2.4. Statistics SPSS 22.0 (IBM Corp., 2013) was used for all statistical analyses. A one-way re- peated measures analysis of variance (ANOVA) was conducted in order to analyse the scores of the CLIL group versus the non-CLIL group on the mathematical test over the three different time points. A Greenhouse-Geisser correction and a Huynh-Feldt correction were used in order to elicit a more accurate and conserva- tive significance value (Abdi, 2010). In addition, ANOVAs were conducted to analyse Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 326 the between-group results of the three different time points (T0, T1, and T2) sep- arately and paired samples t tests were conducted in order to analyse the within- group results separately (T0 compared to T1 and T0 compared to T2 respec- tively). Finally, ANOVAs were conducted to analyse the between-group differ- ence scores (T1-T0 and T2-T0) separately and paired samples t tests were con- ducted in order to analyse the within-group difference scores (T1-T0 compared to T2-T0) for the CLIL group and the non-CLIL group separately. 3. Results 3.1. Between-group and within-group results Table 1 shows the mean scores and standard deviations of the CLIL group versus the non-CLIL group on the mathematical test at the beginning of the study (T0), after three months (T1), and after ten months (T2). At the beginning of the ed- ucation year (T0), the ANOVA revealed that there was no significant difference between the CLIL group and the non-CLIL group on the mathematical test (F(2,105) = .28, p = .60, partial η2 = .003), meaning that the pupils of the CLIL group and the non-CLIL group performed equally well on the mathematical test at the beginning of the study. As can be seen in Table 1 and Figure 1, the one- way repeated measures ANOVA results showed that after three months the CLIL group scored significantly better than the non-CLIL group (F(2,105) = 3.00, p ≤ .05, partial η2 = .055). This difference remained significant from the mean score at the beginning (T0) on the delayed posttest (T3) (F(2,105) = 3.42, p < .05, par- tial η2 = .032), meaning that the pupils of the CLIL group performed better than the pupils of the non-CLIL group after ten months of CLIL education in compari- son with the traditional education (see Figure 1). Table 1 Mean scores (standard deviations) on the mathematical test (maximum score = 100) at the beginning of the study (T0), after three months (T1), and after ten months (T2) Group T0 T1 T2 CLIL group 57.24 (14.03) 67.23ab (13.73) 65.77ac (12.31) Non-CLIL group 58.68 (12.96) 63.05b (15.50) 63.62c (14.89) Notes. a Mean score of the CLIL group is significantly different (p < .05) from the mean score of the non-CLIL group; b mean score after three months (T1) is significantly different (p < .05) from the mean score at the beginning (T0); c mean score after ten months (T2) is significantly different (p < .05) from the mean score at the beginning (T0). The effects of CLIL on mathematical content learning: A longitudinal study 327 Figure 1 Mean scores of the CLIL group (n = 35) versus the non-CLIL group (n = 72) on the mathematical test at the beginning of the study (T0), after three months (T1), and after ten months (T2) Examining the within-group results more closely, both groups showed a sig- nificant improvement of the mathematical test scores after three months (T1) in comparison with the scores at the beginning of the study (T0) (the CLIL group: t = 5.09, p < .001; the non-CLIL group: t = 3.17, p < .01). Moreover, both groups showed a significant improvement of the mathematical test scores after ten months (T2) in comparison with the scores at the beginning of the study (T0) (the CLIL group: t = 4.34, p < .001; the non-CLIL group: t = 3.42, p < .01). Overall, the within-group results showed that both groups scored higher on the mathematical test after following ten months of CLIL or non-CLIL education. 3.2. Between-group and within-group difference scores As can be seen in Table 2, the analysis of the between-group difference scores gives a better insight into the learning curves. Between T0 and T1 (after three months) a significant difference in progression between the CLIL group and the non-CLIL group was found (F(2,105) = 5.48, p < .05, partial η2 = .050), with more progression visible for the CLIL group within the first three months after starting CLIL education in comparison with the non-CLIL group. In contrast, between T1 and T2 (between three and ten months) there was no significant difference in progression between the CLIL group and the non-CLIL group (F(2,105) = 1.81, p Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 328 = .18, partial η2 = .017): Both groups showed the same learning curves and re- mained relatively stable (the CLIL group: t = -1.18, p = .24; the non-CLIL group: t = .66, p = .51). Finally, overall, CLIL pupils’ progress was significantly better over the ten months when compared to their traditionally schooled peers (F(2,105) = 3.42, p < .05, partial η2 = .003). Table 2 Mean difference scores (standard deviations) of the CLIL group (n = 35) ver- sus the non-CLIL group (n = 72) on the mathematical test: (T1)-(T0) and (T2)-(T0) Group (T1-T0) (T2-T0) CLIL group +9.99 (11.62)a +8.53 (11.61) Non-CLIL group +4.37 (11.67) +4.94 (12.26) Note. a Mean difference score of the CLIL group is significantly different (p < .05) from the mean difference score of the non-CLIL group. 4. Discussion In this research, a longitudinal study was performed to find out what the influ- ence of CLIL would be on math performance. There is a growing body of re- search that suggests that CLIL influences cognitive development, which results in better outcomes in science and/or mathematics (Jäppinen, 2005; Murray, 2010; Van de Craen et al., 2007). The first hypothesis of this study was that pu- pils who learn in a CLIL environment are likely to outperform their traditionally schooled peers on mathematical tests, because CLIL creates an increased met- alinguistic awareness. The second hypothesis was that it was unlikely that CLIL students would outperform their traditionally schooled peers after a short pe- riod of time (three months), as the researchers expected that the influence of CLIL would only be noticeable on cognitive tasks after a longer period of time (at least ten months). Here, it is important to note that before the commence- ment of any CLIL education (at T = 0), a pretest showed that the CLIL group and the non-CLIL group were at the same level. After three months (T = 1) and after ten months (T = 2), second and third tests were undertaken to see how the stu- dents’ abilities had evolved. Repeated measures analysis showed that the CLIL group improved significantly better than the non-CLIL group over time. On the third test both groups remained at the approximately the same level they had reached on the second test. Our between-group data is clear proof for the first hypothesis of our study. Pupils who learn in a CLIL environment indeed outperform their traditionally schooled peers on a mathematics test. However, the within-group results reveal that both groups showed an improvement in mathematical performance. The second hypothesis, that the CLIL influence on cognitive level would only be no- ticed after a longer period of time, was proven incorrect because, in contrast to The effects of CLIL on mathematical content learning: A longitudinal study 329 our expectations, even after three months the positive effect of CLIL teaching was clearly noticeable. It was argued that if bilinguals had an increased metalinguistic awareness, which gave them a better understanding and insight into (language) structures (Ter Kuile et al., 2011), the same could be said about pupils who are at a very early stage of second language acquisition and are learning a second language through CLIL. This would not only give them an advantage in understanding the language structures of the “new” language and their mother tongue (Lorenzo et al., 2010; Ter Kuile et al., 2011), but it would also help them to understand the language of mathematics (Arianrhod, 2005; Goldhaber, 2006). This claim is backed up with previous research into the influence of CLIL on mathematics (e.g., Murray, 2010; Van de Craen et al., 2007). 5. Implications The present results confirm the above-mentioned studies on CLIL and mathe- matics and indicate a number of things. First of all, these results add to the re- search body that indicates that CLIL not only positively affects language learning but also content acquisition. CLIL stimulates pupils in such a way that more than one aspect of the learning process is influenced (Surmont et al., 2014). Secondly, it is very surprising to find these positive results already after only three months of CLIL education. It was hypothesized that more time should be given to pupils, at least 10 months upwards, before effects would be found. It would appear that the combination of mathematics and CLIL is so effective that the cognitive stimulation is enormous; however, caution is needed because other possible explaining factors, such as increased motivation, higher self-confidence, and more parental support in the CLIL group were not taken into account (for a detailed discussion of these factors, see Jäkel, 2015). It is recommended that fu- ture research look into the impact of these factors on student outcomes. Thirdly, these data indicate that CLIL possibly influences pupils’ cognitive development, and more specifically, their metalinguistic awareness. This in- creased metalinguistic awareness could lead to a better understanding of and insight into the abstract language of math. Research by Bialystok and Barac (2012) indicates that even in moderate bilinguals an increased metalinguistic awareness can be found, meaning that pupils do not have to be highly proficient in both languages to profit from an increased metalinguistic awareness. In this study pupils were still at a very low level of French proficiency and the results seem to indicate that they indeed already profit from an increased metalinguistic awareness. However, this conclusion has to be approached with caution as an increased metalinguistic awareness has not been directly proven here. Future Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 330 research should therefore look into the impact of CLIL on metalinguistic aware- ness and the question to what extent the advantage (if any) impacts other com- petences such as math achievement. A longitudinal setup in which language proficiency, math achievement and metalinguistic awareness are measured and where variables such as motivation are taken into account should give a clearer insight into this matter. It would also be very interesting to see how different age groups respond (e.g., Jäppinen, 2005) and how much the unique linguistic environment of a specific school (i.e., the native and foreign language back- ground of the teachers, of the other pupils, the location of the school, etc.) af- fects the (possible) effect CLIL has on the cognitive development of the pupils. A final consequence of the present results, and of other results found in the literature (e.g., Adanur, Yagiz, & Işik, 2004; Jäppinen, 2005), is the fact that policy makers should be aware of the link between languages and mathematics. If the European Union intends to improve math performance, it should create bridges between language learning and mathematics (and science for that mat- ter). The present results indicate that a combination of language and content has a positive effect on cognitive development. This indicates that teachers should cooperate with each other and that the schism between language teach- ers and math/science teachers should be overcome. Cooperative learning, both by pupils and teachers, is the way forward. Knowledge is built on previous learn- ing (OECD, 2007), and enabling peer groups to synthesize ideas can create learn- ers with options for accessing learning (Gardner, 1983; Marsh, 2012). This coin- cides with Vygotsky’s (1978) zone of proximal development, where pupils are provided with the necessary help and support for tasks they cannot yet perform on their own. Governments, both local and (trans)national, should increase help for schools which want to introduce CLIL, be it on legislative level or through the provision of enough support and guidance for schools who introduce CLIL. A Eu- ropean framework for CLIL implementation and follow-up could be a possible solution, although in order to succeed, it is important to stop using the current rigid national education systems. Finally, it is important to come to a European agreement on the exact definition of CLIL so that CLIL teachers are able to ben- efit from the experiences and knowledge acquired in various educational set- tings across Europe (Cenoz, Genesee, & Gorter, 2013). 6. Limitation of the study Naturally, the present study has several limitations. One limitation is that there were no standardized math tests for secondary education available at the time of this study. Therefore, three versions of a new mathematical test called the Mathematical Assessment Test-Help had to be developed by the researchers. The effects of CLIL on mathematical content learning: A longitudinal study 331 Although the stimulus material was prepared as carefully as possible, in close collaboration with experts in the field of mathematical education, it would have been better if validated math tests could have been used. A second limitation of the study is the time frame. The students in both the CLIL and non-CLIL group were observed and tested throughout only one ac- ademic year, or ten calendar months. A more interesting test would be to ob- serve the groups over a longer time in order to ascertain whether or not the two groups continued to develop. A more detailed, longer study to compare the CLIL group’s advantages in mathematics over the non-CLIL group and to measure any continued expansion would be hugely beneficial. Thirdly, a specific test of metalinguistic awareness would have been able to test the increased metalinguistic awareness hypothesis. A test such as the Indonesian Language Test (Ter Kuile et al., 2011) or the Wug task (Berko, 1958; Bialystok & Barac, 2012) could have been implemented in order to strengthen the assertion that CLIL education develops such an awareness. This increased metalinguistic awareness, along with increased motivation, higher self-confidence, and more parental support (e.g., Jäkel, 2015) seem to be the factors explaining the results. A specific test of metalinguistic awareness could have proven this more conclusively and could have shed light on the degree to which it is a factor. Moreover, in order to be able to draw firm conclusions, in future research moderator variables (including metalinguistic awareness) should be factored in and controlled for, and multivariate analyses (such as factor or discriminant analyses) should be performed in order to determine which variables (the independent variable, CLIL, or other moderator variables, such as increased metalinguistic awareness) are truly responsible for the better mathematical re- sults of the CLIL group compared to the non-CLIL group. 7. Conclusions The latest results of the Programme for International Student Assessment (PISA) by the Organisation for Economic Cooperation and Development (OECD) have shown that the need to increase mathematic abilities of pupils in Europe is high (European Commission, 2013). The 2020 goal to have a maximum of 15% being low achievers is far off, and the decline in the rate of low achievers is too slow. In this paper, a suggestion was made to take a different route towards better math achievement, namely the CLIL route. The present results confirmed previ- ous research results (e.g., Dawe, 1983; Clarkson, 1992, 2007) showing that CLIL pupils outperformed their traditionally schooled peers in mathematics. The CLIL group’s progress over ten months (one secondary education year) was significantly better than the progress of the traditionally schooled pupils and was already visible Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 332 after a very short period of time (three months). To conclude, these remarkable results shed new light on the math issue and should be taken into account when new math policy is written; however, more research needs to be done in the future, especially over longer periods of time, but also in different contexts and with different age groups. The effects of CLIL on mathematical content learning: A longitudinal study 333 References Abdi, H. (2010). The Greenhouse Geisser Correction. In N. Salkind (Ed.), Encyclo- pedia of research design. Thousand Oaks, CA: Sage. Adanur, Y., Yagiz, O., & Işik, A. (2004). Mathematics and language. Journal of the Korea Society of Mathematical Education Series D: Research in Mathemat- ical Education, 8(1), 31-37. Arianrhod, R. (2005). Einstein’s heroes: Imagining the world through the lan- guage of mathematics. Oxford: Oxford University Press. Banegas, D. L. (2012). Integrating content and language in English language teaching in secondary education: Models, benefits, and challenges. Stud- ies in Second Language Learning and Teaching, 2(1), 111-136. Berko, J. (1958). The child’s learning of English morphology. Word, 14, 150-177. Bialystok, E. (2001). Metalinguistic aspects of bilingual processing. Annual Re- view of Applied Linguistics, 21, 169-181. Bialystok, E., & Barac, R. (2012). Emerging bilingualism: Dissociating advantages for metalinguistic awareness and executive control. Cognition, 122(1), 67-73. Bialystok, E., Peets, K. F., & Moreno, S. (2014). Producing bilinguals through im- mersion education: Development of metalinguistic awareness. Applied Psycholinguistics, 35(1), 177-191. Bonnet, A. (2012). Towards an evidence base for CLIL: How to integrate qualita- tive and quantitative as well as process, product and participant perspec- tives in CLIL research. International CLIL Research Journal, 1(4), 66-78. Bredenbröker, W. (2002). Förderung der fremdsprachlichen Kompetenz durch bilingualen Sachfachunterricht: Empirische Untersuchungen [Improve- ment of foreign-language competency through bilingual instruction: Em- pirical studies]. In S. Breidbach, G. Bach, & D. Wolff (Eds.), Mehrsprachig- keit in Schule und Unterricht: Bilingualer Sachfachunterricht. Didaktik, Lehrer-/Lernerforschung und Bildungspolitik zwischen Theorie und Empi- rie (pp. 141-150). New York: Peter Lang. Cenoz, J., Genesee, F., & Gorter, D. (2013). Critical analysis of CLIL: Taking stock and looking forward. Applied Linguistics, 35(3), 243-262. Clarkson, P. C. (1992). Language and mathematics: A comparison of bilingual and monolingual students of mathematics. Educational Studies in Mathemat- ics, 23(4), 417-429. Clarkson, P. C. (2007). Australian Vietnamese students learning mathematics: High ability bilinguals and their use of their languages. Educational Studies in Mathematics, 64(2), 191-215. Coyle, D. (2006). Content and language integrated learning: Motivating learners and teachers. Scottish Languages Review, 13, 1-18. Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 334 Cummins, J. (2005). Using information technology to create a zone of proximal devel- opment for academic language learning: A critical perspective on trends and possibilities. In C. Davison (Ed.), Information technology and innovation in lan- guage education (pp. 105-126). Hong Kong: University of Hong Kong Press. Dawe, L. (1983). Bilingualism and mathematical reasoning in English as a second language. Educational Studies in Mathematics, 14(4), 325-353. Edwards, H. T., & Kirkpatrick, A. G. (1999). Metalinguistic awareness in children: A de- velopmental progression. Journal of Psycholinguistic Research, 28(4), 313-329. Ellis, R. (2008). The study of second language acquisition. Oxford: Oxford Uni- versity Press. European Commission. (2013). PISA 2012: EU performance and first inferences regard- ing education and training policies in Europe. Brussels: European Commission. Francis, N. (2004). Nonlinear processing as a comprehension strategy: A pro- posed typology for the study of bilingual children’s self-correction of oral reading miscues. Language Awareness, 13(1), 17-33. Freire, P. (1972). Pedagogy of the oppressed. New York: Herder and Herder. Fromkin, V. A. (1980). Errors in linguistic performance: Slip of the tongue, ear, pen and hand. New York: Academic Press. Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic Books. Goldhaber, A. (2006). Math as a language in its own right. American Scientist, 94(2), 1. González, J. Á., & Barbero, J. (2013). Building bridges between different levels of education: Methodological proposals for CLIL at university. Language Value, 5(1), 1-23. Goswami, U. (2008). Cognitive development: The learning brain. New York, NY: Psychology Press. Hermanto, N., Moreno, S., & Bialystok, E. (2012). Linguistic and metalinguistic outcomes of intense immersion education: How bilingual? International Journal of Bilingual Education and Bilingualism, 15(2), 131-145. Horgan, D. (1981). Learning to tell jokes: A case study of metalinguistic abilities. Journal of Child Language, 8(1), 217-224. IBM Corp. (2013). IBM SPSS Statistics for Windows (Version 22.0). Armonk, NY: IBM Corp. Jäkel, N. (2015). Use and impact of language learning strategies on language proficiency: Investigating the impact of individual difference variables and participation in CLIL streams (Unpublished doctoral dissertation). Bo- chum, Ruhr-Universität Bochum. Jäppinen, A. K. (2005). Thinking and content learning of mathematics and sci- ence as cognitional development in content and language integrated learning (CLIL): Teaching through a foreign language in Finland. Language The effects of CLIL on mathematical content learning: A longitudinal study 335 and Education, 19(2), 148-169. Jessner, U. (2006). Linguistic awareness in multilinguals: English as a third lan- guage. Edinburgh: Edinburgh University Press. Kemper, R., & Vernooy, A. (1993). Metalinguistic awareness in first graders: A qualitative perspective. Journal of Psycholinguistic Research, 22(1), 41-57. Lorenzo, F., Casal, S., & Moore, P. (2010). The effects of content and language in- tegrated learning in European education: Key findings from the Andalusian Bilingual Sections Evaluation Project. Applied Linguistics, 31(3), 418-442. Marsh, D. (1994). Bilingual education & content and language integrated learn- ing. Paris: International Association for Cross-Cultural Communication, Language Teaching in the Member States of the European Union (Lingua), University of Sorbonne. Marsh, D. (2012). Content and language integrated learning: A development tra- jectory. Córdoba: Servicio de Publicaciones de la Universidad de Córdoba. Mehisto, P., & Morsh, D. (2011). Approaching the economic, cognitive and health benefits of bilingualism: Fuel for CLIL. In Y. Ruiz de Zarobe, J. Ma- nuel Sierra, & Gallardo del Puerto (Eds.), Content and foreign language integrated learning: Contributions to multilingualism in European contexts (pp. 21-48). Bern: Peter Lang. Mondt, K., Struys, E., Balériaux, D., & Van de Craen, P. (2008). Neurocognitive effects of multilingualism and school language on numerical cognition in a population of children. NeuroImage, 41(Suppl. 1). Murray, D. R. (2010). Irish-medium language immersion programs’ effect on mathematics education. Journal of Mathematics Education at Teachers College, 1, 28-32. Nagy, W. E., & Anderson, R. C. (1995). Metalinguistic awareness and literacy ac- quisition in different languages (Tech. Rep. No. 618). College of Education, Univeristy of Illinois at Urbana-Champaign, USA. Nagy, W. E., & Anderson, R. C. (1998). Metalinguistic awareness and the acquisition of literacy in different languages. In D. Wagner, R. Venezky, & B. Street (Eds.), Literacy: An international handbook (pp. 155-160). Boulder, CO: Westview. Nikula, T., Dalton-Puffer, C., & García, A. L. (2013). CLIL classroom discourse: Re- search from Europe. Journal of Immersion and Content-Based Language Education, 1(1), 70-100. OECD. (2007). Understanding the brain: The birth of a learning science. Paris: Organisation for Economic Cooperation and Development. Orr, E. W. (1987). Twice as less: Black English and the performance of black stu- dents in mathematics and science. New York: Norton & Company. Pérez-Cañado, M. L. (2012). CLIL research in Europe: Past, present, and future. In- ternational Journal of Bilingual Education and Bilingualism, 15(3), 315-341. Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 336 Ransdell, S., Barbier, M., & Niit, T. (2006). Metacognitions about language skill and working memory among monolingual and bilingual college students: When does multilingualism matter? The International Journal of Bilingual Education and Bilingualism, 9(6), 728-741. Rumlich, D. (2013). Students’ general English proficiency prior to CLIL: Empirical evidence for substantial differences between prospective CLIL and non- CLIL students in Germany. In S. Breidbach & B. Viebrock (Eds.), Mehrspra- chigkeit in Schule und Unterricht: Content and language integrated learn- ing (CLIL) in Europe. Research perspectives on policy and practice (pp. 181- 201). Frankfurt am Main: Lang. Sun, R., Mathews, R., & Lane, S. (2007). Implicit and explicit processes in the development of cognitive skills: A theoretical interpretation with some practical implications for science education. In E. Vargios (Ed.), Educa- tional psychology research focus (pp. 1-26). Hauppauge, NY: Nova Science. Surmont, J., Van de Craen, P., Struys, E., & Somers, T. (2014). Evaluating a CLIL student: Where to find the CLIL advantage. In R. Breeze, C. Llamas Saiz, C. Martinez Paramar, & C. Tabernero Sala (Eds.), Integration of theory and practice in CLIL (pp. 55-74). Amsterdam: Rodopi. Ter Kuile, H., Veldhuis, M., Van Veen, S. C., & Wicherts, J. M. (2011). Bilingual education, metalinguistic awareness and the understanding of an un- known language. Bilingualism: Language and Cognition, 14(2), 233-242. Thompson, D. R., & Rubenstein, R. N. (2000). Learning mathematics vocabulary: Poten- tial pitfalls and instructional strategies. Mathematics Teacher, 93(7), 568-574. Usiskin, Z. (1996). Mathematics as a language. In P. C. Elliott & M. J. Kenney (Eds.), Communication in mathematics, K-12 and beyond (pp. 231-243). Reston, VA: National Council of Teachers of Mathematics. Valladoloid, L. (1991). The effect of bilingual education on student’s academic achievement as they progress through a bilingual program (Unpublished doctoral dissertation). United States International University, Nairobi, Kenya. Van de Craen, P., Ceuleers, E., & Mondt, K. (2007). Cognitive development and bilingualism in primary schools: Teaching maths in a CLIL environment. In D. Marsh & D. Wolff (Eds.), Diverse contexts – converging goals: CLIL in Europe (pp. 185-200). Frankfurt am Main: Peter Lang. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Whitehurst, G. J., & Lonigan, C. J. (1998). Child development and emergent lit- eracy. Child Development, 69(3), 848-872. World Medical Organization. (1996). Declaration of Helsinki. British Medical Journal, 313(7070), 1448-1449. The effects of CLIL on mathematical content learning: A longitudinal study 337 Supporting information Additional supporting information may be found in the online version of this article at the journal’s website: http://ssllt.amu.edu.pl/download/docs/SSLLT%206(2) %20Surmont,%20Struys,%20Van%20Den%20Noort,%20Van%20De%20Craen%20 Appendices.pdf.