Disa...... 10 Creation is distributed under the Creative Commons License Attribution-Share Alike 4.0 International Published in: http://ejournal.stkipmpringsewu-lpg.ac.id/index.php/smart Jurnal SMART : Journal of English Language Teaching and Applied Linguistics. BEHIND THE IMPROVEMENT OF STUDENTS’ ENGLISH PROFICIENCY IN AN EMI UNIVERSITY Disa Evawani Lestari President University disa.silaen@president.ac.id Abstract This study aims to examine the improvement rate of students after their one- year study at an EMI university as well as to further explore the challenges and the opportunities they encounter throughout the period. To answer those questions, 100 first-year students from one EMI university were recruited as research participantsthrough a purposive sampling method: 20 participants from A1 - C1 level. The participants were assigned to do a standardized test prior to the start of their study (pre-test) and after their one-year study (post- test). The improvement rate was analyzed quantitatively using T-tests. Data about the challenges and opportunities they encounter to improve their English proficiency was collected through a questionnaire and analyzed qualitatively. The result shows that the improvement of English proficiency is experienced by all groups (A1-C1), with B2 group who get the highest improvement. From the questionnaire, it is found out that writing skill gets the least improvement.Thefactors that promote and impede their progress are also discussed. This gives valuable information which can affect English teaching and learning materials and methodology in order to maximize the opportunities of studying in an EMI university. Keywords: Challenges, EMI, English proficiency, improvement; opportunities. 1. Introduction In the globalization era in which English proficiency determines the competitiveness of an individual or a country, education institutions, especially the higher education ones, are thriving to make sure that their graduates can compete in the global market. One of the policies which is applied to increase graduates’ competitiveness is implementing English as a Medium of Instruction (EMI) policy. EMI, which is also referred to as bilingual immersion and content & language integrated learning (Dearden, 2014) is often practiced to meet this goal. Jurnal SMART , Volume 6, No 1 (2020), Page. 10-23 ISSN Cetak : 2356-2048 ISSN Online : 2356-203X DOI: 10.26638/js.981.203X mailto:disa.silaen@president.ac.id Behind the improvement..... 11 EMI policy has become a significant educational trend (Graddol, 2006) in many countries because being proficient in English is considered as a capital to modernize and develop a country (Hu, 2008) and that English is the language of the top-level business, tourism and education (Vinke, Snippe and Jochems, 1998). The decisions behind the implementation of EMI by many educational institutions are often triggered by the economic concept of globalisation (Collins, 2010) which promises the best opportunities and academic and career advancement for those with proficient English (Crystal, 2012). Dearden (2014) outlined that there are several important reasons why there is a marked rise in the emergence of EMI policy in educational institutions in ESL & EFL countries. To ESL countries, implementing EMI is deemed as a way a university can maximize subject integration opportunities, enhance education level to compete in a global market as well as attract more international and exchange students to study in their institution. On the other hand, the reason behind the implementation of EMI in EFL countries appear to be more about language acquisition. In Indonesia and Japan, this policy is to raise foreign language skills so that the students can compete in a global market. The implementation of EMI in EFL countries are often influenced by the requirement of learning in an English-speaking country. It can be frequently seen that institutions applying EMI put the requirement of a certain TOEFL or IELTS score as one of its admission requirements as it is practiced by the universities in English- speaking countries to their international applicants. Studies in the field of language learning and teaching supports the idea that a second language is learned most effectively when learners are exposed to sufficient input of the target language. Intense exposure to English, which is promised by the implementation of EMI, becomes the rationale underpinning this practice. It is largely based on the idea that second language acquisition is similar to first language acquisition. The rationale for EMI in university level is often based on theories of acquisition which support a naturalistic process of language learning similar to first language acquisition in which learning takes place effortlessly and automatically, with the requirements that there is sufficient exposure to English and the learner is sufficiently motivated feels the need to do it. Disa...... 12 Research on EMI has indicated that its implementation has caused improvement on various aspects of language competence, including lexis (Lo & Murphy 2010), speaking (Ruiz de Zarobe 2008), and writing (Lasagabaster 2008). These studies have generally found advantages of CLIL programs over non-CLIL ones. For example, Ruiz de Zarobe (2008) found that the CLIL students outperformed the non-CLIL in pronunciation, vocabulary, grammar, fluency, and content. While there has been an extensive number of studies and a general belief that the implementation of EMI policy can improve English proficiency, only limited study (Lei & Hu, 2014) explores what actually happen during the period of exposing learners to English-speaking environment. This is the gap that this study aims to fill in: explore the opportunities & challenges learners encounter to inform a more effective and responsive teaching and learning policy, method, and material in order to maximize the potential of learning English in an EMI university. In addition to that, this study also aims at measuring the improvement rate among learners with different English proficiency levels to provide extra information for the basis of setting a certain TOEFL or IELTS score as a student admission requirement to an EMI university which is mostly adopted from the implementation TOELF or IELTS score requirement in English-speaking country. To address those goals, the following research questions are formulated: (1) To what extent does initial language proficiency affect the improvement rate of students in one year?; (2). What are the opportunities and challenges learners encounter during their first year of study in an EMI university? 2. Research Methods This study applied mixed-method for the data analysis. The quantitative research was applied to measure the improvement of participants’ language proficiency over a one-year period. On the other hand, the qualitative research was applied to gain in- depth responses from the participants about the challenges and opportunities they encounter regarding the development of their language proficiency while studying in an EMI university. The participants of this study were 100 first-year students in an EMI university in Indonesia from all study programs available in that university, ranging from International Relations, Communications, Elementary Teacher Education, Actuarial Science, Accounting, Management to Engineering study programs. All of Behind the improvement..... 13 those 100 participants were recruited through a purposive-sampling method grouped based on their initial TOEFL test score acquired during the university English placement mapping. 20 participants represented each of the CEFR levels of A1, A2, B1, B2, and C1 group. This purposive sampling method was applied to ensure that each group was represented by an equal number of participants and, therefore, the improvement rate among those groups are valid. The instruments used in this study were a pre-test document (TOEIC test), a post-test document (TOEFL test), and an online questionnaire. TOEIC test was chosen as a post-test document because of considering that not all freshmen came from an English-speaking university. Thus, a more general English test would be more accurate to capture the participants’ language proficiency level prior to the start of their study and an academic English test, like a TOEFL test, would be better used to measure the participants’ English proficiency level after their one-year exposure to an EMI university. To measure the improvement rate of their pre-test and post-test score, the pre-test score (TOEIC score) was first converted to a TOEFL score using the Conversion Table below. Finally, correlation test was conducted to measure the improvement rate of their Pre-TOEFL score and their post-TOEFL score. Table 1. TOEFL Conversion table Disa...... 14 The online questionnaire consists of several likert-scale statements to measure their perception towards the improvement they think they have acquired on four language skills (Speaking, Listening, reading, and Writing). In addition to that, some open-ended questions were designed to capture their responses on what are the factors that give significant impact to the improvement of their English proficiency. Data collection happened in two stages: 1. Stage1:Pre-test,beforethestartoftheacademicyear(September2018) 2. Stage2:Post-test,aftercompletingaone-yearstudy(September2019) A TOEIC test was conducted in stage 1, with coordination with President Development Centre. A TOEFL test was conducted in stage 2 along with the distribution of online questionnaire at the end of the test. Data was analysed both quantitatively and qualitatively. The results of the participants pre-test and post-test were compiled and the improvement rate of each group (A1-C1) was measured by conducting a T-test. On the other hand, the responses on the online questionnaire were analysed qualitatively. 3. Findings and Discussion To answer the first research question, the improvement rate of each group wasmeasuredby doing a correlation test (T-test) toward the pre-test and post-test score. The results of each group were presented below. Improvement Rate of A1 Table 2. A1 Paired Sample Statistics Mean N Std. Deviation Std. Error Mean Pair 1 PreA1 333.2000 20 3.99473 .89325 PostA1 356.2500 20 29.03152 6.49164 Based on the table above, the average of pre-test in A1 is 333.2. After the implementation of the method, the average score of A1 improves to 366.25. The improvement (23.05 points) should be tested by paired sample t-test to see whether the improvement is significant or not. The paired samples t-test results may show in the table below. Behind the improvement..... 15 Table 3. A1 Paired Sample Test Paired Differences t df Sig. (2- tailed) Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper Pair 1 PreA1 - PostA1 -33.05000 27.38127 6.12264 -45.86483 -20.23517 -5.398 19 .000 The table above shows that t-score is -5.398 that is significant in error level of 0.000. This level is lower than maximum error level that this research may tolerate: 0.05. Based on this result, it can be summarized that there is significant improvement of TOEFL Test score before and afteraction. Improvement Rate of A2 Table 4 A2 Paired Sample Statistics Mean N Std. Deviation Std. Error Mean Pair1 PreA2 PostA2 410.2500 419.7000 20 20 31.86794 47.87385 7.12589 10.70492 Based on the table above, the average of pre-test in A2 is 410.25. After the implementation of the method, the average score of A2 improves to 419.7. The improvement (9.45 points) seems too small, and should be tested by paired sample t-test to see whether the improvement is significant or not. The paired samples t-test results may show in the table below. Table 5. A2 Paired Sample Test Paired Differen ces t df Sig. (2- taile d) Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper Pair1 PreA2 - PostA2 -9.45000 27.68521 6.19060 -22.40708 3.50708 -1.527 19 .1 43 Disa...... 16 The table above shows that t-score is -1.527 that is significant in error level of 0.143. This level is higher than maximum error level that this research may tolerate: 0.05. Based on this result, it can be summarized that there is nosignificant improvement of TOEFL Test score before and after action. Or the action did not improve the TOEFL score in A2 Group. Improvement Rate of B1 Table 6. B1 Paired Sample Statistics Mean N Std. Deviation Std. Error Mean Pair1 PreB1 PostB1 459.1000 479.2000 20 20 2.40394 31.34964 .53754 7.00999 Based on the table above, the average of pre-test in B1 is459.10. After the implementation of the method, the average score of B1 improves to 479.2. The improvement (20.1 points) should be tested by paired sample t-test to see if the improvement is significant or not. The paired samples t-test results may show in the following table. Table 7. B1 Paired Sample Test Paired Differences t df Sig. (2- tailed) Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper Pair1 PreB1 – PostB1 -20.10000 30.60598 6.84371 -34.42404 -5.77596 -2.937 19 .008 The table above shows that t-score is -2.937 that is significant in error level of 0.008. This level is lower than maximum error level that this research may tolerate: 0.05. Based on this result, it can be summarized that there is significant improvement of TOEFL Test score before and after action in B1 Group. In other words, B1 group get Behind the improvement..... 17 the most advantages of English proficiency improvement. Their English proficiency improves significantly just by spending one year studying at an EMI university. Improvement Rate of B2 Table 8. B2 Paired Sample Statistics Mean N Std. Deviation Std. Error Mean Pair 1 PreB2 PostB2 518.0500 543.4500 20 20 14.83763 24.47227 3.31779 5.47217 Based on the table above, the average of pre-test in B2 is518.05. After the implementation of the method, the average score of B2 improved to 543.45. The improvement (25.40 points) should be tested by paired sample t-test to make see whether the improvement is significant or not. The paired samples t-test results may show in the following table. Table 9. B2 Paired Sample Test Paired Differences t df Sig. (2- tailed) Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper Pair1 PreB2 – PostB2 -25.40000 19.90610 4.45114 -34.71634 -16.08366 -5.706 19 .000 The table above shows that t-score is -5.706 that is significant in error level of 0.000. This level is lower than maximum error level that this research may tolerate: 0.05. Based on this result, it can be summarized that there is significant improvement of TOEFL Test score before and after action in B2Group. Improvement Rate of C1 Table 10. C1 Paired Sample Statistics Mean N Std. Deviation Std. Error Mean Pair 1 PreC1 PostC1 571.9500 574.1000 20 20 18.18176 27.99229 4.06556 6.25927 Disa...... 18 Based on the table above, the average of pre-test in C1 is571.95. After the implementation of the method, the average score of C1 improved to 574.1. The improvement (2.15 points) seems too small, and should be tested by paired sample t-test to see if the improvement is significant or not. The paired samples t-test results may show in the table below. Table 11. C1 Paired Sample Test The table above shows that t-score is -0.408 that is significant in error level of 0.688. This level is higher than maximum error level that this research may tolerate: 0.05. Based on this result, it can be summarized that there is nosignificant improvement of TOEFL Test score before and after action. Or the action was not improved the TOEFL score in C1Group. Overall Result Table 12. Improvement Rate among Groups Group TOEFL Improvement t-value Sig Summary A1 23.05 -5.398 0.000 Significantly Improved A2 9.45 -1.527 0.143 No Improvement B1 20.1 -2.937 0.008 Significantly Improved B2 25.40 -5.706 0.000 Significantly Improved C1 2.15 -0.408 0.688 No Improvement Based on the table above, the biggest improvement is B2 Group. It is proven by t-test result that shows that B2 t-score is the highest. Therefore, it can be summarized that the method was most effective for B2 Students Characteristics. The improvement in B2 is in line with language theories which require a certain threshold of English score to Paired Difference s t df Sig. (2- tailed) Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper Pair1 PreC1- PostC1 -2.15000 23.55794 5.26771 -13.17545 8.87545 -.408 19 .688 Behind the improvement..... 19 get admitted in an EMI university (Coleman, 2006). Interestingly, not only proficient users who seem to benefit from the EMI policy. In the table above, it can be seen that the A1 group also benefit significantly from the EMI environment to a percentage of improvement which is really close to that experienced by B2 group.Thismerits further research. Learners’ Perceptions on their English Improvement To answer the second research question, learners’ perceptions on their English improvement are collected through a likert-scale which ranges from 1 to 5, with 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, and 5=strongly agree. Table 13. Perceptions on English Improvement Based on Skills Skills Agree/ Strongly Agree Neutral Disagree/ Strongly Disagree Speaking 69% 23% 8% Listening 62% 28% 10% Reading 69% 23% 8% Writing 60% 35% 5% From the table above, it is clearly seen that the skill which the participants reported to have improved most significantly are Speaking and Reading. Also, the data show that two-thirds of the participants perceive that improvement happen to the four language skills, with Writing skill considered as the area with the least improvement compared to the other three skills. It is further captured in the open-ended question that writing is the most difficult skill to improve. This could be linked to their difficulty in reading academic textbooks. In one of the Likert-scale, it was found out that almost one-third of the participants (31%) claimed that it was difficult for them to understand written course textbooks or academic materials and almost one-third of them (30%) were ambivalent. The difficulty of grasping knowledge in a scientific and academic language was experienced by more than half of the participants. This can be seen in the following graphic. Disa...... 20 Graphic 1. The difficulty of grasping knowledge Factors Supporting Improvement of English Proficiency There are three major factors which play an important role in the improvement of learners’ English proficiency, they are lecturers, major association (BEM) and students’ club and communities. It is interesting to see that the factors which support the improvement of their English proficiency lie on their interaction (active engagement) with their teachers and peers rather than their exposure to English language materials in textbooks, movies, games,etc. This can be observed in the following figure. Figure 1. Factors Supporting Improvement of English Proficiency Furthermore, the open-ended questions reveal that the advantage they get out of studying in an EMI university is higher motivation and confidence to use English. They admit that they get more motivation to improve their English because it has become a need and that they also see that they are not the only one who got average English. This Behind the improvement..... 21 is in line with a study conducted by Moratinos-Johnston, Juan-Garau& Salazar- Noguera(2018). The other advantage recurrently reported by the participants is feedback from peers and lecturers. It seems that students feel positive towards feedback. Instead of seeing it as a criticism, many of them admit that feedback is something they expect to get especially from lecturers to improve their English proficiency. Factors Impeding Improvement of English Proficiency The first most significant factors which participants reported to impede the improvement of their English proficiency is students’ preference to speak Indonesian language out of classrooms and during group discussion, especially when the presence of a lecturer is not there to help them when they do not know the words or feel uncertain about the grammar of their sentence. It implies that students grouping should consider the existence of the more proficient students to help their less proficient peers when using English outside classrooms. Another challenge reported by participants are errors spoken by their friends. These were reported to trigger them to get trapped into making the same error. Errors are often ignored, not corrected as they do not always feel good correcting their friends’ English especially when errors happen here and there. It can be understood that error correction is an important thing to give as its absence can lead to the downgrade of English proficiency (Amara, 2015; Erdogan, 2005). This gives implication to further study which focus on training students several ways of giving error correction. In addition to that, there should also be a culture promoted by the stakeholder to maintain students’ positive view towards correction, not as something demotivation or humiliating but as something constructive. 4. CONCLUSION This study confirms the existing research on how intense exposure to English language use accelerate the improvement of English proficiency. More interestingly, this study has expanded the scope of the implementation of EMI. It has been widely researched that only proficient users are considered able to function well and to benefit from an English-speaking institution. Those studies might back up the policy behind making a certain TOEFL or IELTS score as the requirement for university admission. Disa...... 22 However, this study reveals that even the non-proficient user of English benefit significantly from the EMI policy. the result shows that the A1 group benefit significantly from the EMI environment indicated by the improvement rate of that group which is really close the improvement rate experienced by the B2 group. it indicates that there could be some major differences of the implementation of EMI policy in an English-speaking country with that in a non-English-speaking country. This merits further research. In addition to that, more focus should be given to the development of English proficiency which involves human interaction. It is found out that learners benefit a lot from their interaction with their teachers and peers rather than the printed materials. It is recommended that EMI university provides a student’s club or language centre which could be an informal way of students to acquire English through its use in a daily communication. The implication of how online platform could accommodate these needs merit further research. 5. REFERENCES Amara, N. (2015) Error corrections in foreign language teaching. The Online Journal of New Horizons in Education, 5(3), 58-68. Collins, A.B. (2010). English-medium higher education: Dilemma and problems. Eurasian Journal of Educational Research, 39, 97-110. Crystal, D. (2012). English as a global language. Cambridge university press. Dearden, J. (2014). English as a medium of instruction-a growing global phenomenon. British Council. Erdoğan, V. (2005). Contribution of error analysis to foreign language teaching. Mersin University Journal of the Faculty of Education, 1(2), 261-270. Graddol, D. (2006). English next: Why global English may mean the end of ‘English as a Foreign Language. London: The British Council. Hu, G. (2008). The misleading academic discourse on Chinese-English bilingual education in China. Review of Educational Research, 78, 195-231. Lasagabaster, D. (2008). Foreign language competence in content and language integrated learning. Open Applied Linguistics Journal, 1, 31–42. Behind the improvement..... 23 Lei, J., & Hu, G. (2014). Is English-medium instruction effective in improving Chinese undergraduate students' English competence?. International Review of Applied Linguistics in Language Teaching, 52(2), 99-126. Lo, Y-Y., and Murphy, V. A. (2010). Vocabulary knowledge and growth in immersion and regular language-learning programmes in Hong Kong. Language and Education, 24, 215–238. Moratinos-Johnston, S., Juan-Garau, M., & Salazar-Noguera, J. (2018). The effects of English-medium instruction in higher education on students’ perceived level and self-confidence in ELF. Learning context effects, 75. Ruiz de Zarobe, Y. (2008). CLIL and foreign language learning: A longitudinal study in the Basque country. International CLIL Research Journal, 1, 60–73. Vinke, A.A., Snippe, J., &Jochems, W. (1998). English-medium content courses in Non-English higher education: A study of lecturer experiences and teaching behaviours. Teaching in Higher Education, 3(3), 383-394.