Successful integration of refugee students in higher education 158 ______________________________________________________________________________________________________________________ Global Education Review is a publication of The School of Education at Mercy College, New York. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC by 4.0), permitting all use, distribution, and reproduction in any medium, provided the original work is properly cited, a link to the license is provided, and you indicate if changes were made. Citation: Zlatkin-Troitschanskaia, Olga; Happ, Roland; Nell-Müller, Sarah; Deribo, Tobias; Reinhardt, Franziska; & Toepper, Miriam. (2018). Successful integration of refugee students in higher education: Insights from entry diagnostics in an online study program. Global Education Review, 5 (4), 158-181. Successful Integration of Refugee Students in Higher Education: Insights from Entry Diagnostics in an Online Study Program Olga Zlatkin-Troitschanskaia Johannes Gutenberg University Mainz (Germany) Roland Happ Johannes Gutenberg University Mainz (Germany) Sarah Nell-Müller Johannes Gutenberg University Mainz (Germany) Tobias Deribo Johannes Gutenberg University Mainz (Germany) Franziska Reinhardt Johannes Gutenberg University Mainz (Germany) Miriam Toepper Johannes Gutenberg University Mainz (Germany) Abstract Accessing higher education without having to overcome bureaucratic hurdles is a serious concern for refugees. Although empirical studies on the integration and success of refugees in higher education are scarce, the challenges related to this issue are becoming apparent. The Success and Opportunities for Refugees in Higher Education (SUCCESS) research project has been launched to investigate the effectiveness of new online study programs offered on the Kiron Open Higher Education (Kiron) platform that provides refugees with access to Massive Open Online Courses (MOOCs). SUCCESS measures the prior knowledge and skills of refugee students and investigates to what extent their study opportunities, learning processes, and chances of academic success can be improved effectively through different forms of support provided in Kiron. In this paper, we present the assessment framework and study design of the SUCCESS project as well as data on 1,376 students entering the study program in Kiron in summer 2017. As students’ language skills, intellectual abilities, and prior study-related knowledge play a significant role in their performance in higher education degree programs, we focus on the crucial introductory study phase and valid diagnostics of students’ study preconditions. We analyze refugee students’ socio- biographical and educational data such as gender, country of origin, highest level of education achieved etc. and examine their English language skills, intellectual abilities, and previous study domain related knowledge. We find extreme differences in levels of education and preconditions on starting to study in Kiron. Based on these results, we discuss implications for the effective and successful integration of refugee students in higher education. Keywords Integration of refugees, higher education, study success, Massive Open Online Courses 159 Global Education Review 5(4) Project Objectives and Research Focus Participation in higher education programs is considered an effective way to integrate refugees into their host country (Colucci et al., 2017; De Wit & Altbach, 2016; Zorlu, 2013). Although studies that focus on the integration of refugees in higher education are scarce, the challenges arising from these issues in actual practice are evident (Crea, 2016; Larsen, Kornbeck, Kristensen, Larsen, & Sommersel, 2013; Morrice, 2013). Many host countries (like Kenya) lack the resources and capacities for higher and professional education, which is therefore often neglected (e.g., Wright & Plasterer, 2012). Moreover, refugees also face numerous obstacles when attempting to enter higher education study programs in countries with well-established state-run higher education systems, for example European countries in general, and Germany in particular1. Available studies indicate several of these factors at various levels, such as legal and formal difficulties (e.g., laws, missing documents), language barriers, and/or lack of money (e.g., Lorisika, Cremonini, & Safar Jalani, 2015). Research on academic success indicates that refugee students are more prone to dropping out of university study programs than non-refugee students (Arnold, 2013; Duong, Badaly, Liu, Schartz, & McCarty, 2016; van Herpen, Meeuwisse, Hofman, Severiens, & Arends, 2017; Zorlu, 2013); disadvantageous higher education entry conditions, such as linguistic deficits in the language of instruction or deficits in terms of study-related prior education play a crucial role in facilitating these high drop-out rates (Ben-Moshe, Bertone, & Grossman, 2008; Jungblut & Pietkiewicz, 2017; Mendenhall, Russell, & Bruckner, 2017). Kiron Open Higher Education (Kiron; https://kiron.ngo/) is a non-profit ed-tech organization that was established in 2015 in order to circumvent some of the main global obstacles refugees often face when entering higher education in their host countries. Kiron seeks to provide refugees direct, unbureaucratic, and free access to higher education through digital solutions and by supporting their academic success. Kiron’s online study program is based on a digital and fully modularized curriculum clustering Massive Open Online Courses (MOOCs) offered via Kiron’s learning platform Kiron Campus and its ecosystem of support services (e.g., language courses). The overarching aim of Kiron is to enable refugees to begin studying in their host countries as quickly as possible. At Kiron, prospective students with an asylum or refugee status, from all countries across the globe can register online at any time. After joining the platform and completing a questionnaire on their educational backgrounds and an English language test, registered Kiron students can choose one of five study tracks: Mechanical Engineering, Business and Economics, Computer Science, Political Science, or Social Work. After approximately one or two years of online learning, students can apply to one of Kiron’s partner universities, where – upon acceptance – they can complete their bachelor’s degree in a regular (offline) degree program. To this end, Kiron has closed agreements with a large network of partner universities worldwide, which will award up to 60 credits for completed Kiron online modules according to the standards of the European Credit Transfer and Accumulation System (ECTS) (for more details, see Rampelt & Suter, 2017). A new research project, Success and Opportunities for Refugees in Higher Education (SUCCESS), was conducted by the German Federal Ministry of Education and Research (BMBF) in early 2017 in order to examine the Corresponding Author Olga Zlatkin-Troitschanskaia, Department of Business and Economics Education, Johannes Gutenberg University, Saarstraße 21, 55122 Mainz, Germany Email: troitschanskaia@uni-mainz.de Successful integration of refugee students in higher education 160 impact of this promising approach to enhancing the successful integration of refugee students in higher education in their host countries worldwide. The SUCCESS project is scheduled to last for three years and to analyze (longitudinally) the teaching-learning progress of Kiron students from the time they register and start studying on the Kiron platform (usually up to two years) until they transfer to the regular higher education system in the host countries. In the SUCCESS project, individual, institutional, and external contextual factors have been identified that promote or impede the successful integration of refugee students in higher education through the Kiron platform (see Section 2). Based on the results of SUCCESS, recommendations could be derived for politics and practice in higher education. Even though digital learning is being discussed as a promising approach in the current integration debate (e.g., Colluci et al., 2017), there is very little evidence of its factual effectiveness to date. Using Kiron as an example, the SUCCESS project aims to conduct important empirical investigations and provide insights into potentials, but also to highlight the limitations of approaches of this kind for the successful integration of refugees. Insights gained into how refugees can study effectively through digital solutions and how they can be transferred successfully to regular universities in host countries can guide educational and integrational practice. Findings from research on academic success and evidence from higher educational practice demonstrate that students’ prior education (Dochy, Segers, & Bühl, 1999; Richardson, Abraham, & Bond, 2012; Robbins et al., 2004), study-related preconditions such as language skills (Demie & Strand, 2006; Fakeye & Ogunsiji, 2009) and previous knowledge play a significant role in their performance in higher education degree programs (Brand & Xie, 2010; Rienties, Beausaert, Grohnert, Niemantsverdriet, & Kommers, 2012). Research on academic performance among refugee students in particular highlights extreme heterogeneity in students’ prior education, language proficiency, general intellectual abilities, study-related domain-specific knowledge, and their major impact on academic success (Brückner et al., 2015; Callahan & Humphries, 2016; Griga, 2014; Robbins et al., 2004). Accordingly, preconditions of this kind must be assessed and considered in an objective and valid manner (AERA, APA & NCME, 2014) in order to successfully integrate refugees. Diagnostic assessments during the introductory phase (prior to beginning to study) are therefore crucial for providing refugee students with suitable study recommendations, appropriate domain-specific study opportunities such as online courses, and effective additional support measures, for example, in terms of language (Crea, 2016; Joyce, Earnest, De Mori, & Silvagni, 2010). In this context, assessment-based feedback for the students is of particular importance and it is also an enormous challenge considering the extreme heterogeneity of the learning-relevant, socio-cultural backgrounds of the students. In addition to the individual study and learning recommendations (following the idea of a triangle model by Pellegrino, Chudowsky, & Glaser, 2001), consequences for the design of curricula and for the instruction of students have been drawn from the assessments. This enables provision of the best possible and individually tailored teaching-learning environment for refugee students, and helps to prepare them for successful transfer to the regular higher education system. In this paper, we focus on this crucial introductory phase and the accompanying valid diagnostics of students’ preconditions when starting to study on Kiron. To this end, Kiron students who registered on Kiron Campus between May and September 2017 additionally completed an intelligence (IQ) test and a domain-specific test in their chosen study track (see Section 3) as part of the SUCCESS project. 161 Global Education Review 5(4) In Section 2 we describe the study framework; by first defining the underlying framework model and our evaluation approach and then outlining the SUCCESS assessment framework based on them. In Section 3, we present descriptive results from the empirical analysis conducted during the registration process of the 1,376 students enrolled in a study program on Kiron. We analyze the personal data provided in the questionnaire, such as gender, country of origin, and highest level of education obtained, and determine the refugee students’ English language skills and study-related previous knowledge when starting to study on Kiron (see Section 4). The findings from the analysis of student-related data obtained during the registration process and the additional assessments should provide indications of how to provide these students with the best possible support regarding their individual preconditions. Thanks to valid and reliable assessments of students’ language skills, level of education, and previous domain-specific knowledge when beginning to study on Kiron, it could be determined which learning opportunities are suitable for each student and what kind of additional support measures (e.g., individual mentoring, a personal buddy) would effectively facilitate the integration of refugee students into regular higher education. Finally, the limitations of the study are discussed and an outlook for further research is outlined (see Section 5). Study Framework The SUCCESS project’s evaluation approach is in line with the evidence-centered design (ECD) by Mislevy and Haertel (2006), as we attempt to deduce valid interpretations for students and educators from the gathered empirical results (see also the standards for pedagogical and educational testing by the AERA et al., 2014, and their defined validity criteria). In order to achieve this aim, based on current educational research on academic success – under particular consideration of studies with refugee students including the pioneering studies of educational integration of refugees (e.g., Dryden-Peterson, 2016; Schroeder & Seukwa, 2017) and of their academic success (e.g., Benseman, 2014; Harris & Marlowe, 2011; van Herpen et al., 2017) – we have developed an assessment framework for the SUCCESS project. First, we differentiate between theoretically potential influential factors on the study success at various levels, which we define as independent variables: 1) external study circumstances in the host countries (such as internet access, computers etc., for more details see Reinhardt et al., 2018); 2) institutional study factors (such as chosen degree course, teaching- learning-modules etc.), and 3) individual students’ preconditions. Second, we identify and assess a number of indicators of academic success and failure, which we define as dependent variables. These include both objective data provided by Kiron on the refugees’ study activity on the Kiron platform, started and completed teaching- learning-modules, etc. (see Reinhardt et al., 2018), as well as results of the assessment of students’ knowledge and skills while learning on Kiron. To this end, in the SUCCESS project, the students’ knowledge and skills when starting to study on Kiron are assessed using internationally validated test instruments in an objective, valid, and reliable manner and the development of students’ domain-specific knowledge across the different phases of their studies is traced (see Section 3 & 4). A particular focus of the SUCCESS project is on Kiron students who drop out of their learning programs and their reasons for doing so (for models of dropping out of education, see Horn & Kojaku, 2001; Kuh, Kinzie, Buckley, Bridges, & Hayek, 2007). For this purpose, an additional questionnaire is administered and interviews are conducted with a sub-sample in the SUCCESS project. In this collaborative SUCCESS project, Kiron is responsible for Successful integration of refugee students in higher education 162 Figure 1. Evaluation model in the SUCCESS project. recruiting refugee students, familiarizing them with the project, coordinating and performing data collection for analyses, as well as communicating the project results to the students to stimulate individual reflection on their learning progress. Project partner Johannes Gutenberg University Mainz (JGU) coordinates the SUCCESS project and evaluates both the effectiveness of Kiron’s online educational model as an innovative blended learning approach as well as support measures for refugee students in order to prepare Kiron students for the higher education system and regular degree programs at partner universities3. Accordingly, the SUCCESS evaluation model comprises several phases (see Figure 1). A multi-level analysis that integrates numerous methods is conducted over two study phases in the SUCCESS project: the first on Kiron and the second at partner universities. The study design encompasses 1) status analyses when students start studying on Kiron, 2) formative assessment analyses during the two study phases, and 3) summative analyses at the end of studies. The combined longitudinal and cross- sectional design (Dielman, 1989) involves test- based analyses of learning progress on the Kiron platform as well as surveys completed by students at three points during their studies: upon enrollment, one year after starting to study, and when transferring to a partner university in one of many countries (e.g., Germany, France, Sweden, Turkey, Jordan) where Kiron students can complete their bachelor’s degree. Investigating the effectiveness also involves analyzing the efficacy of individual measures implemented in the project such as language courses. Overall, the empirical results should indicate the efficacy of the online courses based on digital MOOCs and measures implemented in Kiron over the course of the SUCCESS project – making a valuable contribution to the extremely limited research in this field – and provide implications for stakeholders involved in higher education practice and politics. In this paper, we focus on analyzing the preconditions of refugees entering higher education on Kiron. We present results from the empirical analysis conducted during the onboarding process of refugees into the Kiron program as well as from additional entry test- based assessments. In addition to analyzing the background questionnaire, including socio- demographical factors and self-reported education levels, we analyze the following academic preconditions: level of English skills, general intellectual abilities and previous domain-specific knowledge. 163 Global Education Review 5(4) Data and Instruments The data was collected in the SUCCESS project from the assessments conducted with newly registered Kiron students during the summer term of 2017 and includes data from 1,376 refugees. For all test instruments, an English version was used, seeing as all courses and communication on the Kiron platform take place in English as well. Moreover, it is Kiron’s goal to achieve a transfer of refugee students to partnering higher education institutions in the host countries (see Section 2), which in turn use English (besides their national language) as their teaching language. That is why refugees’ language skills were tested directly when entering into Kiron, so that possible language- related effects on the performance in other tests (such as domain-specific tests) can be controlled. Additionally, short, linguistically precise expressions were used in the selection of tests as well as in test instructions. All knowledge tests were pretested with Kiron students who were not part of the SUCCESS cohort (see Section 4). During registration on Kiron, all prospective students were asked to fill in a questionnaire on their social and biographical background, with only few compulsory items. Participation in all additional, test-based assessments was voluntary. As an incentive, students were offered individual feedback on their test results that included recommendation and support regarding the selection of their courses on the Kiron platform, a participation certificate as well as a participation fee of 10 Euros. Educational Background To assess the level of self-reported secondary and tertiary education achieved in the country of origin, the International Standard Classification of Education (ISCED; UNESCO, 2012) of the United Nations Educational, Scientific and Cultural Organization’s (UNESCO) was applied. The educational level achieved in the national, formal institutions – from the elementary sector to doctoral programs – is assigned an ISCED- level from 0 to 8, which makes it internationally comparable. The current version includes criteria for allocation of any national, formal education program to the different levels (Eurostat, 2016).4 General English Language Proficiency Assessments of the students’ General English Language Proficiency were conducted using a C- test (Grotjahn, 1987; Norris, 2006). The C-Test is an economically efficient instrument for assessing General Language Proficiency (GLP) in foreign, second and first languages (Harsch & Hartig, 2015; Grotjahn, 2002). It is based on the phonics-centered model from literacy assessment research (Freeman & Freeman, 2000).5 A C-test comprises several short texts in which the participants have to fill in missing halves of words that have been deleted beforehand according to certain principles). The C-Test used in our study contains four short texts that are self-contained and suitable for different language levels (Harsch, Tschauschew, & Brandt, o.J., p. 3). In the four texts with 23-29 items, more than 100 words must be completed based on content and grammar in the space of 20 minutes (ebd., p. 3f.). The C-Test used in SUCCESS was specifically adapted for Kiron (Suter, Harsch, & Brandt, 2017). The results gathered in the C-test were modeled using the Rasch model (de Ayala, 2009; Embretson & Reise, 2000). By including certificates of the Internet-based Test of English as a Foreign Language (TOEFL iBT; no older than two years) from refugees who want to study on the Kiron platform, it was possible to establish a link between the results of the C-test and the Test of English as a Foreign Language score (TOEFL, Educational Testing Service, 2008) within the SUCCESS cohort, and to then match the C-test score to the Common European Framework of Reference for Languages metric (CEFR; Hawkins & Filipović, 2012; Little, 2007; North, 2014; Verhelst, van Successful integration of refugee students in higher education 164 Avermaet, Takala, Figueras, & North, 2009). Therefore, the results of the language test become comparable on an internationally recognized scale. The CEFR metric describes language proficiency at six levels: A1 and A2: Basic User A1: Beginner; A2: Elementary English; B1 and B2: Independent User B1: Intermediate English; B2: Upper-Intermediate English; C1 and C2: Proficient User: C1: Advanced English; C2: Proficiency English. General Cognitive Ability To assess the students’ generic intellectual skills, a subtest on figural intelligence of the Intelligenz-Struktur Test (I-S-T 2000R; Liepmann, Beauducel, Brocke, & Amthauer, 2007) was used as a relatively language- independent indicator.6 The I-S-T 2000R is based on the hierarchical framework model from intelligence structure research. It differentiates between seven primary factors according to Thurstone (1938) and specifies, on the hierarchical level, fluid and crystallized intelligence according to Cattell (1963). The basic module of the I-S-T 2000R consists of nine groups of tasks, which assess verbal, numerical, and figural intelligence, as well as deductive thinking, and two further task groups that assess retention (Liepmann et al., 2007). In the study, a subtest was created from the basic module of the I-S-T 2000R, which, using the task group ‘figure choice’, enables researchers to make statements about figural-spatial intelligence. Participants were given seven minutes to complete all 20 task items. Since this task group is based on deductive thinking, which is considered an appropriate indicator for general intelligence, the choice of this subtest seems reasonable in order to assess the cognitive ability of the participants. Moreover, this task group is nonverbal and can also be classed as culturally neutral. Intrinsic and Extrinsic Study Motivation Study motivation was assessed using scales from the questionnaire on academic interest (FSI; Schiefele, Krapp, Wild, & Winteler, 1993). In the SUCCESS project, we use the validated short scales with four items each and four Likert- scaled response options (Biewen, Happ, Schmidt, & Zlatkin-Troitschanskaia, 2018). The results of the tasks in question serve not only as a scale to describe (occupational) extrinsic motivation but also to assess study-related intrinsic motivation. Study Track Specific Knowledge Tests The findings on study-related previous knowledge presented here are based on domain- specific knowledge tests. During the selection of the tests that would be most suitable for this study, special attention was paid to whether the test contents align with both the contents on the platform Kiron and the core curricula taught at the partnering universities in the major host countries. Consequently, there may be countries among the refugees’ countries of origin whose school or tertiary education content in, for example, business and economics, may deviate from these curricular structures or contents. However, as the study’s aim is to assess refugees’ entry preconditions when entering into the Kiron platform and partnering higher education institutions in the host countries, the contents taught there were used as guidelines for our assessments. The test instruments are meant to assess the refugee students’ prior knowledge in relation to these targeted academic contents, irrespective of the refugees’ countries of origin. Therefore, we made sure for all five study tracks that the contents assessed in the tests correspond to the contents taught on the Kiron platform and the partnering universities. During selection of the domain-specific knowledge tests, curricular analyses of the content of the Kiron online courses were conducted in the five study tracks in order to ensure curricular and instructional validity (Pellegrino, 2016). Additionally, analyses of the regular university degree programs in these areas in the major host countries were conducted. While a large common denominator in the sense of an 165 Global Education Review 5(4) internationally valid core curriculum in the domain of economics became apparent, strong differences also became evident in other subject areas, such as Social Work in particular. Therefore, the expert ratings of (solely German) university lecturers were included in this selection process. These experts work at the partnering universities the Kiron students in Germany aim to be eventually transferred to. A short version with 15 items from the internationally established US-American Test of Economic Literacy (TEL; 4th ed.) of the Council for Economic Education (CEE) (Walstad, Rebeck, & Butters, 2013a) was used to assess students’ previous knowledge in the study track Business and Economics. The economic content in the TEL is based on the CEE Standards (2010), which is an internationally established description of what expert economists and economic educators consider the core economic concepts and principles to be taught to pre- college students in the OECD countries (Walstad et al., 2013a: 301). Studies from different countries worldwide indicate that the TEL reliably and validly assesses the economic prior knowledge at the beginning of studies (for the US, see Walstad et al., 2013b; for Germany, see Happ, Förster, Zlatkin-Troitschanskaia, & Carstensen, 2016; for Japan, see Yamaoka, Asano, & Abe, 2010; for Korea, see Jahn, Hahn, & Kim, 2010). Previous knowledge in Mechanical Engineering was assessed using a short version of the Representational Variant of the Force Concept Inventory (R-FCI; Nieminen, Savinainen, & Virri, 2010). The Force Concept Inventory (FCI) (Hestenes, Wells, & Swackhamer, 1992) measures R-FCI students’ ability to interpret multiple representations (i.e., representational consistency) in the context of forces (Nieminen, 2013). According to Nieminen et al., (2010), the test’s reliability and validity of measuring young adults’ knowledge of physics can be rated as good. The R-FGI comprises 27 items. For the SUCCESS sample, 18 items were selected according to the abovementioned selection criteria and in consultation with the test developers. A short-version of the Computertest für die Personalauswahl [computer test for personnel recruitment] (C-PA; Wagener, 2013), with 24 adapted items translated into English, was used to measure previous knowledge in Computer Science. The C-PA assesses knowledge in dealing with computers, which is divided into the four areas application, Internet, hardware and computer science (Wagener, 2003). The German original version of C-PA was translated and adapted by the project team with translation experts according to the Translation, Review, Adjudication, Pre-testing and Documentation (TRAPD; Harkness, 2003) approach, whereby the Test Adaption Guidelines (TAGs) (ITC, 2017) were also taken into account when adapting the English instrument. To assess previous knowledge in Social Work, 12 items from the Knowledge Mental Illness Test (MC-KOMIT; Furnham, Gee, & Weis, 2016) were used. The test comprises mental health literacy. Of the initial 33 items, 12 items that have a particularly strong correlation with the curricular content of the study track Social Work (e.g., attention deficit hyperactivity disorder, Alzheimer's disease or substance abuse) were selected. Previous knowledge in Political Science was assessed through 15 economic policy items of the TEL (Walstad et al., 2013a). Additionally, Political Interest and Internal Political Efficacy (Niemi, Craig, & Mattei, 1991) was measured. Political Interest of the SUCCESS sample was assessed using a translated short scale of the Zentralarchiv für empirische Sozialforschung (ZA) und Zentrum für Umfragen, Methoden und Analysen (ZUMA) (ZA & ZUMA, 2014). The English short version of the Political Efficacy (PEKS; Beierlein, Kemper, Kovaleva, & Rammstedt, 2014) was used to assess the Internal Political Efficacy of the sample7. Successful integration of refugee students in higher education 166 Analysis and Results In this paper, the focus is on the descriptive analyses of students’ background data within the SUCCESS project sample gathered from the onboarding questionnaire upon registering on the Kiron platform. The analyses include socio- biographical data as well as data on English language proficiency, general cognitive ability and domain-specific knowledge. As the participation in all tests is voluntary, the data is in part characterized by a relatively high number of missing values documented below (see also Reinhardt et al., 2018). Socio-biographical Background The following information was self-reported by students over the Kiron platform during the registration process.8 Analysis of students’ responses in the questionnaire reveals extreme diversity in their socio-biographical backgrounds. The SUCCESS cohort comprises 1,376 refugees from 54 different countries of origin: about half of the sample comes from the Syrian Arab Republic (37%), Somalia (8%) and Afghanistan (6%); the other half of the students come from other African, Arab or Asian countries. Furthermore, the SUCCESS sample resides in 66 different host countries. The majority of the sample are located in the following host countries: Germany (28%), Jordan (18%), Turkey (13%), Kenya (8%), and France (7%). These host countries are also among the ten largest host countries as determined by the United Nations (UNHCR, 2017). From the SUCCESS sample, the majority of refugees from Africa reside in neighboring countries (such as Kenya) (see also UNHCR, 2012). The average age of students in the SUCCESS sample is M = 28.55 (SD = 6.4) years. The age range in the sample is between 15 and 61 years. For comparison, the median age of immigrants to Europe in 2015 was 27.5 years (Eurostat, 2017). In the SUCCESS project sample, 20% of Kiron students are female. This percentage is lower than that of female refugees in Germany (in 2017, 26% of refugees aged 18– 25 and 32% aged 25–30 were female, see BAMF, 2017b) and in Europe. In Europe, the percentage of female refugees who applied for asylum was 32.7% in 2016 (Castella, 2017). This percentage of females studying on Kiron also strongly deviates from the proportional distribution of students in regular degree programs in Germany where, on average, 50% of students in the higher education sector are female (BiBB, 2017). The comparison with the descriptive statistics for the complete Kiron cohort (N=3000) shows that the SUCCESS sample, which makes up approximately 50% of all Kiron students, could be considered representative for all Kiron students. However, this only applies to the socio-biographical data and not the results of the additional tests presented here. As the participation in all assessments is voluntary, a positive self-selection of, for instance, particularly high-performing students cannot be ruled out, as underlined by the results of test motivation in assessment research. Furthermore, it can also be assumed that the Kiron students differ from the general refugee cohort, as having an interest in an online degree course only addresses specific a target group. Moreover, the distinct differences in the distribution of gender and the students’ current country of residence as seen in the official statistics mentioned above suggest that representativeness of the sample for the general refugee cohort cannot be assumed and should therefore be critically examined in future studies (see Section 5). Educational Level Following the ISCED level, the self-reported data indicate a high average level of education for the sample. Three quarters of the sample (74%, n = 1,018) state upper secondary education as their highest level of education. Only 6% of Kiron students (n = 84) reported a school leaving certificate under secondary education. Almost half of the students (53%) stated they had 167 Global Education Review 5(4) already studied at different tertiary levels. Remarkably, 71% of students with tertiary experience reported to have graduated, mostly with a degree equivalent to a bachelor’s degree (56%). However, 29% of students had to terminate their degree programs at various points. The reported average length of time spend in tertiary education was 3.5 semesters (SD = 2.3), despite a bachelor’s degree typically lasting 6–8 semesters (OECD, Eurostat, & UNESCO Institute for Statistics, 2015). Only 5% of the students have a master’s degree or equivalent level of education, and 1% completed their studies with a doctoral degree. An additional 9% claimed to have completed various forms of academic education such as post-secondary non-tertiary education or short- cycle tertiary education. Overall, almost one third of the students in the sample claimed to have tertiary education experience at different levels. Kiron students with completed degree programs mostly studied subjects such as Social Science, Business and Law (27%; n = 140), followed by Engineering, Manufacturing and Construction (23%; n = 118) and Science, Mathematics and Computing at 17% (n = 88) ab. In the case of uncompleted degrees, the subjects were mostly Social Sciences, Business and Law (27%; n = 95), followed by Engineering, Manufacturing and Construction (23%; n = 83) and Science, Mathematics and Computing (20%, n = 77). General English Language Proficiency Although over a third of study participants had reported English as a common language in their previous education, overall the results of the language test showed major weaknesses in the Kiron students’ language proficiency. The language test takers (i.e., 1,169 of the 1,376 students in the SUCCESS project sample9) exhibited great variance in their General English Language Proficiency.10 In the sample, the test takers achieve M = 440 (SD = 140), with a minimum of 42 and a maximum of 862 points. Compared to the standard values (M = 500; SD = 100), the achieved mean value is in the range of 1 SD below the standard values. For a quantitative classification of the C-test scores within the CEFR levels presented above, see Table 1. On the 95%-confidence-interval (CI), only 0 to 6% of the Kiron students in the project sample had a C1 language proficiency level and 9% to 32% had B2. Consequently, more than 60% of the sample does not have the B2 or C1 level required for university (academic education). In addition, there is a marked discrepancy between the self-reported and Table 1 Estimated and self-reported CEFR Levels N A2 or lower (%) B1 (%) B2 (%) C1 or higher (%) Estimated CEFR Level 1,157 261 (22.56) 674 (58.25) 204 (17.63) 18 (1.56) Estimated CEFR Level (Min 95%) 1,126 573 (50.89) 457 (40.59) 96 (8.53) 0 Estimated CEFR Level (Max 95%) 1,157 0 717 (61.97) 366 (31.62) 74 (6.40) Self-reported 1,203 118 (9.81) 347 (28.84) 442 (36.74) 296 (24.61) Successful integration of refugee students in higher education 168 actual (test-based) level of language proficiency: 90% of the students estimate their English language level to be B1 or higher. Nearly one third (27%) even estimate their own English level to be C1 or higher. General Cognitive Ability The test-takers (n = 631 of the 1,376 students in the SUCCESS project sample) achieve an average of M = 8.3 (SD = 3.4) of 20 possible points, which corresponds to a standard value of 96 IQ points in the 26–30-year-old age group. The results of the general level of cognitive ability are thus slightly below average. In comparison with other findings from the I-S-T 2000R, this demonstrates clear differences in the test result of more than one standard deviation (e.g., Bühner, Ziegler, Krumm, & Schmidt-Atzert, 2006). The reliability of the general ability scale with Cronbach's α = 67, however, indicates that other indicators for general cognitive abilities should be considered for further analyses. Study Motivation Generally, the Kiron students indicated a high degree of motivation to study, regardless of age, gender or self-reported level of education. The results (n = 1,106 of the 1,376 students in the SUCCESS project sample) demonstrate high values on a four-Likert scale on intrinsic (M = 3.4; SD = .5) and extrinsic study motivation (M = 3.3; SD = .6). The extent to which a high level of motivation is regarded as a protective factor against study failure should be investigated in future analyses (see Section 5). Domain-specific Knowledge The domain-specific knowledge tests were first used in a pretest and then in the main test.11 A comparison of the pretests and main test results is presented in Table 2. Based on the results of the pretesting, the domain-specific tests were partially slightly adapted in the main test: Due to the high total score and the item difficulties encountered, a more difficult version of the C-PA was used in the main test. The order of the Political Science test was changed (from easy items to difficult items). The tests for Social Work, Business and Economics and Mechanical Engineering remained unchanged compared to pretesting. The results of the domain-specific pretests indicate that the pretested Kiron students have a solid level of previous knowledge in Computer Science; approximately half of all items or more were solved correctly in Business and Economics and Political Science. In Social Work and Mechanical Engineering, study-related knowledge levels were rather low on average. The results of the SUCCESS cohort were somewhat worse in all study tracks compared to pretesting, as, on average, less than half the tasks were solved correctly. While the SUCCESS cohort was tested when they started to study, the Kiron students chosen for the pretest stage were previously registered Kiron students. Therefore, the slightly worse performance of the SUCCESS cohort compared to pretesting is in line with expectations. Additionally, the study track Political Science assessed the Political Interest and Political Efficacy of the SUCCESS cohort. Overall, the students showed high Political Interest (M = 4; SD = 1) and high Political Efficacy (M = 4.1; SD = 1.1). To analyze the scale’s reliability, the two measures for the internal consistence of Cronbachs α and McDonalds ω (McDonald, 1999; for a differentiation between ωt and ωh, see Revelle & Zinbarg, 2009) were calculated. In view of research underlining the limitations and bias of the α-coefficient (e.g., Revelle & Zinbarg, 2009; Sijtsma, 2009; Sijtsma & van der Ark, 2015), the values of the ω-coefficient should be weighted higher in the interpretation of the findings. Overall, the test procedures used showed satisfactory internal consistencies (Cronbach’s α of .69 – .87). In the TEL and MC- KOMIT, the McDonald’s Omega as an additional estimator compared to alpha indicated good internal consistence. 169 Global Education Review 5(4) Table 2 A comparison of the pretests and main test results Study tracks Pretests SUCCESS Cohort N = Num. of students 12 n = valid questionnaire N = Num. of students n = valid questionnaire Business & Economics max. test score: 15 N = 327 n = 53 N = 415 n = 66 M (SD) 8.5 (2.8) 6.7 (2.6) Cronbach’s α .70 .53 McDonald’s ωt .75 .65 Computer Science max. test score: 24 N = 271 ne = 90 N = 505 nd = 63 M (SD) 18.4 (3.9) E 12.8 (5.4) D Cronbach’s α .79 .87 McDonald’s ωt .82 .90 Political Science max. test score: 15 N = 253 n = 30 N = 144 n = 23 M (SD) 7 (3.4) 5.7 (1.7) Cronbach’s α .73 .68 McDonald’s ωt .81 .78 Mechanical Engineering max. test score: 18 N = 151 n = 17 N = 108 n = 19 M (SD) 6.4 (2.9) 5.8 (3.4) Cronbach’s α .76 .70 McDonald‘s ωt .87 .83 Social Work max. test score: 12 N =307 n = 35 N = 204 n = 25 M (SD) 4.1 (5.7) 3.7 (2.7) Cronbach’s α .58 .72 McDonald’s ωt .71 .83 Note: M = Mean; SD = Standard deviation, ne = easy short version of C-PA, nd = difficult short version of C-PA Discussion and Conclusion Critical Interpretation of the Results As the state of research (see Section 1) indicates the crucial importance of prospective students’ entry requirements for successful admission and academic success in tertiary education (e.g., Kuh et al., 2007), the analysis in this paper focuses on the descriptive analyses of the cognitive and non-cognitive entry (pre)conditions of refugee students when starting their study track on the online-based learning platform Kiron. The Kiron student sample represents 54 different countries of origin. The students are currently located in Successful integration of refugee students in higher education 170 66 different host countries. This indicates an enormous origin-related diversity of refugee students as well as extreme heterogeneity of the external framework conditions that refugee students are faced with while studying online on Kiron and upon transitioning to regular higher education in the host countries. Remarkably, half of the SUCCESS sample is located in host countries with a high gross domestic product (GDP). It is evident that countries with a high GDP have different external framework study conditions than countries with a lower GDP (e.g., Reinhardt et al., 2018). Age distribution, with an average of 28.5 (SD = 6.4) years as well as high study motivation values for both intrinsic and extrinsic motivation (Tohido & Jabbari, 2012) indicate that Kiron has a large number of refugees for whom access to tertiary education might be a crucial step towards integration into the host society (see also Harris & Marlowe, 2011; Zeus, 2011). This is also indicated by the data on educational backgrounds and previous education, according to which most Kiron students have not only completed high school education and are therefore entitled to higher education studies (see also UNESCO, 2017), but also that a large part of the sample has already begun or even completed higher education. Approaches such as those applied on Kiron could offer refugees the opportunity to successfully continue their educational paths that were interrupted by having to flee from their countries of origin, and to realize their educational and professional goals in the host countries. From both economic and humanitarian perspectives, this is immensely important not only for the individuals themselves but also for the host society (in case of long-term settlement) and for the country of origin in case refugees are able to return at a later point. Integration into higher education can also reduce the risk of a so-called ‘lost generation’ (De Wit & Altbach, 2016; see also Reinhardt et al., 2018). In addition to the necessity of effective integration approaches for refugees in higher education, the findings from the test-based assessment of study preconditions of refugee students indicate that many areas regarding a potentially successful degree course often exhibit major apparent deficits, and that special support measures are therefore urgently required for a successful integration of refugees. The results of the C-test on General English Language Proficiency demonstrate that only 2% of the respondents reach level C1 and 18% reach level B2. Accordingly, approximately three-quarters of the respondents are below the level of B2 recommended for access to higher education. Thus, language deficits represent an enormous restriction for participation in the teaching- and- learning opportunities and achievement of academic success on Kiron and in the host countries. One approach for the integration of refugees into tertiary education is therefore language promotion among refugees. This should include teaching-learning courses for effectively acquiring a second and third language, such as English and the host country's language, as well as the promotion of native language skills at an academic educational level (see also Reinhardt et al., 2018). Considering the I-S-T 2000R results as a measure of cognitive ability, it should be emphasized that general intellectual ability is typically regarded as an important prerequisite for academic success (e.g., Duckworth, Quinn, Lynam, Loeber, & Stouthamer-Loeber, 2011). According to the study success research, general cognitive ability is related to linguistic performance as well as domain-specific cognitive achievements (e.g., Kaplan, Stolk, Valibhoy, Tucker, & Baker, 2016). Especially in the case of refugee students, cognitive ability is of importance, not only for successful educational processes but also for the integration process (e.g., Pagel, Richter, & Schupp, 2018). For instance, it is assumed that general cognitive ability could play a compensatory role in the event of linguistic or subject-specific deficits; an impairment of cognitive ability is associated with higher drop-out rates in educational programs 171 Global Education Review 5(4) among refugees (e.g., Kira, Lewandowski, Yoon, Somers, & Chiodo, 2012). For the findings on general cognitive ability determined by means of the I-S-T 2000R sub-test, it is critical to note that the sub-test covers only one aspect of intelligence (see Section 3). Therefore, these test results only provide an indicator of the general intelligence of the test persons, which is not enough to make a prediction about the academic success of the refugees. However, this indicator can serve as one of the control variables in examining the test results for linguistic performance and domain- specific previous knowledge. Beyond biographical and cognitive factors, volitional factors also play an important role in academic achievement. They influence learning behaviour and study success (Krapp, 1999; Zheng, Rosson, Shih, & Carroll, 2015), particularly in higher education and in digital online study programs (e.g., Dewitte & Lens, 2000; Husman & Corno, 2010). So far, only the Kiron students’ intrinsic and extrinsic study motivation was assessed in the SUCCESS project. In the research on study success, many other self-regulating skills were identified as influencing factors (e.g., interest, self-efficacy) (e.g., Kuh et al., 2007; Larsen et al., 2013), the importance of which should also be investigated in terms of successful integration and study success of refugee students. In addition to the general cognitive and motivational (non-cognitive) entry requirements of students, it is especially the subject-specific prior knowledge that is an essential factor for academic success in higher education (e.g., Kuh et al., 2007). Remarkably, the findings in the SUCCESS sample vary between the students in the five different study tracks. While the Kiron students in the study tracks Computer Science and Business and Economics have, on average, a solid level of study-related domain-specific knowledge, the students in the study tracks Social Work and Mechanical Engineering demonstrate partially high deficits, which require subject-specific support measures for successful integration and completion of the degree course. In terms of the development of the teaching-learning programs for refugee students, the findings of these assessments highlight far-reaching implications for the design of curricula and instructions on the Kiron platform, which should be implemented at all different levels and address students with very different levels of knowledge. In the further analyses, there needs to be an investigation of the extent to which such subject-specific deficits can also be successfully remedied through digital online learning, so that students with unfavourable entry requirements can also transfer to regular educational programs in the host countries after one to two years of studying in Kiron. Overall, the findings reported in this paper demonstrate that the group of academically- interested refugees on the Kiron platform is extremely heterogeneous. This results from the examination of several factors such as the socio- biographical and educational backgrounds, English language skills, study motivation and domain-specific knowledge. This heterogeneity indicates specific challenges regarding the successful integration of refugees into higher education. It implies that students with a refugee background need more individual, appropriate, and specific approaches and measures to promote their integration into higher education and their general academic success, including the development of their personal and corresponding learning- and study-related values and attitudes. Limitations The evidence presented here has some limitations. For instance, the high percentage of higher degrees among the Kiron students indicates a potentially “positively” self-selected sample. Since the results presented here are based on the responses from students who voluntarily participated in the SUCCESS project assessments, it can be assumed that the Kiron Successful integration of refugee students in higher education 172 students who participated in the study tend to be highly motivated and high-performing, as several studies on students’ willingness to participate in tests indicate. Hence, a positive self-selection of the subsamples for the assessments cannot be ruled out, which is probably on average characterized by slightly more favorable entry requirements than the overall student population on the Kiron platform. Moreover, a positive self-selection can also not be ruled out in view of all Kiron students, as particularly highly motivated and high-performing students tend to be more interested in this online higher education program. Compared to other studies with refugees, there are systematic differences (Chopra & Adelmann, 2017) as more young adults with a refugee background have less than an upper secondary level education. Another selection from the SUCCESS sample is the unequal distribution among both the different countries of origin and the host countries. Compared to the refugee situation worldwide (UNCHR, 2017), the distribution among the host countries is only comparable to a limited extent. Although the host countries of the SUCCESS sample are also among the 10 most common countries, half of Kiron students are hosted in relatively high-income countries. In comparison, around 84% of the refugees worldwide live in low- and middle-income countries; in Africa in particular, refugees tend to move to neighbouring countries. These often have the status of developing countries themselves (UNCHR, 2017). The integration and support of refugees is therefore not the same in the different host countries and a comparable study situation is not guaranteed for all refugee students. Further analyses indicate statistically significant differences between various groups of refugee students with respect to different external study conditions in the SUCCESS cohort’s host countries and also depending on their country of origin (see Reinhardt et al., 2018). Further, the self-reported data of the socio-biographical and educational background show a high number of missing values (up to 20%). It can be assumed that the missing values in the data are not random, but might follow a pattern. Studies indicate that students with an unfavourable expected test performance tend to avoid these test instruments as opposed to students with an expected high-test performance (e.g., Duffy, Smith, Terhanian, & Bremer, 2005). It therefore can be assumed that those refuge students for whom there is no available information on their prior knowledge, cognitive ability or language skills would achieve on average lower results in the tests. Considering the self-reported nature of the data on educational backgrounds, plausibility in general can be critically questioned on the basis of response tendencies such as social desirability (on differences between self-assessment and the use of test instruments, see Swope & Schmitt, 2006). As the test-based results from the C-test demonstrate, the refugee students overestimate their language skills, which becomes particularly clear upon analysis of the self-reported data on General English Language Proficiency in which almost one quarter (25%) of the data is allocated to C1 and more than one third (37%) of the data is at B2. The objective indicators of study-relevant knowledge, intellectual ability and linguistic proficiency reported in this paper, based on validated test instruments, thus make an indispensable contribution to valid and reliable entry diagnostics. It is, however, questionable to what extent the students can provide reliable information at all, particularly in view of the high discrepancies in self-assessment and test- based results of language skills. Perhaps students assess their language skills according to other benchmarks and criteria such as every-day spoken language. It can also not be assumed that all students can properly apply standards like the CEFR levels. At the same time, however, the discrepancies that arise between self-reported and achieved test values in the language test can 173 Global Education Review 5(4) also provide students with important impulses in determining their individual weaknesses for which targeted support can be provided in the host countries, including helping students choose suitable remedial offers such as language courses. Finally, it should be critically remarked with regard to the findings presented in this paper, that all test instruments were provided to participants in English, as the Kiron learning program and all communication on the platform is also provided in English. A transfer to partnering higher education institutions typically requires certification of English proficiency at a B2 level where the teaching language is typically English (or the respective national language). As the results of the language proficiency test indicate considerable deficits for several participants, it can be assumed that using English as the assessment language has affected overall test performance. It is possible that participants might have performed better if the knowledge tests had been administered in their native language or the higher education language of their country of origin. Implications The results from the SUCCESS project are intended to enhance an understanding of how to successfully integrate refugees into higher education and society, and shall form an evidence-based foundation for higher education policy makers and stakeholders to foster preparation of refugees for accredited degree programs and promote academic success. As outlined in Section 1, the successful integration of refugees in the education sector is a vital task that every country taking in refugees is faced with. The findings of the SUCCESS project indicate that the challenges of successfully integrating refugees and fostering their academic success differ greatly from the challenges of integrating students with migration backgrounds and promoting their academic success (e.g., Rienties et al., 2012). While migrants generally consciously prepare for integration, refugees do not usually leave their country of origin voluntarily and are accordingly unprepared (e.g., leaving without relevant documents, no competence in the language of the host country, Connor, 2010). When compared with many measures for the successful integration of refugees into higher education (see Section 1), one strength of the SUCCESS project lies in the fact that objective and valid entry diagnostics based on test instruments were already gathered at the beginning of studies, on the basis of which participants are given individual recommendations regarding their course selection on the Kiron platform and/or transfer to the partnering higher education institutions. If and to what extent Kiron students are actually successful in their studies on the Kiron platform and upon transfer to the partnering universities as well as whether and to what extent this is affected by their internal preconditions will be examined further during the course of the SUCCESS project using process diagnostics (see Section 2). As external individual framework conditions in the host countries can also differ greatly (Reinhardt et al., 2018), their impact on the academic success of refugees should also be considered for the successful integration of refugees. Studies of this kind allow for evidence- based insights on effective support measures for refugee students. Acknowledgements 1. The SUCCESS project is being funded by the German Federal Ministry of Education and Research under the grant number 16HLQ007. 2. We would like to thank our project partner Kiron and particularly Renata Suter and Steffen Brandt for their cooperation in this study. 3. We would like to thank the two anonymous reviewers who provided constructive feedback and helpful guidance in the revision of this paper. Successful integration of refugee students in higher education 174 Notes 1. Nearly 750,000 applications for asylum in Germany were filed in 2017, more than ¾ of which were filed by refugees under the age of 30 (BAMF, 2017a). 2. Additionally, the SUCCESS project uses data gathered through online self-assessments developed by the RWTH Aachen as consulting instruments and sources of information on the refugee students. The Munich University of Applied Sciences examines further complementary support. For more details see https://success.uni- mainz.de/. 3. Informally acquired qualifications that are not formally verifiable may not be taken into account. 4. It should be critically noted that current research is at a disagreement as to which language assessment framework is best. The literacy assessment models contain multiple elements such as semantic, syntactic, pragmatic, and phonemic awareness. The General English Language Proficiency Assessment C-Test used in this study strictly focuses on the phonemic-centered model. In view of existing research on multilingual learners (Freeman & Freeman, 2000; Krashen, & McField, 2006; Moll, 1992), this poses a limitation to our study. 5. The instructions for the tasks of the subtest on figural intelligence were only provided in English, which may affect the test results. However, this only concerns a few sentences; the test itself, as a subtest on figural intelligence, is viewed in literature as relatively language-independent and therefore particularly suitable for a target sample with language barriers (e.g., migrants). 6. We refrain from administering the PEKS items on External Political Efficacy (Balch, 1974) for research ethical reasons. 7. The following information may be distorted by socially desirable response behavior 8. As in all other tests, participation in the C- test was not obligatory and approx. 200 Kiron students did not take part in this test. 9. Participation in all tests of the SUCCESS project is voluntary. The tests are low-stakes rather than high-stakes. In low-stakes testing, participants’ motivation to take the test can vary greatly, which in turn can partially restrict the validity of the interpretation of the test results (for consequences of low-stakes vs. high-stakes testing, see e.g., Haertel, 1999; Rios & Liu, 2017; Stenlund, Lyrén, & Eklöf, 2017; for language testing in particular, see Schmidgall & Powers, 2017). 10. The pretest students are not included in the SUCCESS cohort and began studying on the Kiron platform prior to the summer term of 2017. 11. All refugee students were contacted via the platform Kiron Campus and asked to complete an online questionnaire on a voluntary basis. All students were sent a specific questionnaire according to the study track they are enrolled in on Kiron Campus. Accordingly, n = "enrolled students" indicates the number of students who received the survey link through their Kiron platform email address. The indication n= "valid questionnaires" represents the number of students who completed the questionnaire. This assessment design highlights that a positive self-selection of participation can be expected. References American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association. 175 Global Education Review 5(4) Arnold, I. J. M. (2013). Ethnic Minority Dropout in Economics. Journal of Further and Higher Education, 37(3), 297–320. Balch, G. I. (1974). Multiple indicators in survey research: The concept “sense of political efficacy”. Political Methodology, 1, 1–43. Beierlein, C., Kemper, C. J., Kovaleva, A., & Rammstedt, B. (2014). Political Efficacy Kurzskala (PEKS). Zusammenstellung sozialwissenschaftlicher Items und Skalen. doi:10.6102/zis34 Ben-Moshe, D., Bertone, S., & Grossman, M. (2008). Refugee access and participation in tertiary education and training. Melbourne: Institute for Communitiy, Ethnicity and Policy Alternatives (ICEPA), Victoria University. Benseman, J. (2014). Adult Refugee Learners with Limited Literacy: Needs and Effective Responses. Refuge, 30(1), 93–103. Biewen, M., Happ, R., Schmidt, S., & Zlatkin- Troitschanskaia, O. (2018). Knowledge Growth, Academic Beliefs and Motivation of Students in Business and Economics – A longitudinal German Case Study. Higher Education Studies, 8(2), 9–28. Brand, J. E., & Xie, Y. (2010). Who Benefits Most from College? Evidence for Negative Selection in Heterogeneous Economic Returns to Higher Education. American Sociological Review, 75(2), 273–302. Brückner, S., Förster, M., Zlatkin- Troitschanskaia, O., Happ, R., Walstad, W. B., Yamaoka, M., & Asano, T. (2015). Gender Effects in Assessment of Economic Knowledge and Understanding: Differences Among Undergraduate Business and Economics Students in Germany, Japan, and the United States. Peabody Journal of Education, 90(4), 503–518. Bundesamt für Migration und Flüchtlinge (BAMF) (2017a). Aktuelle Zahlen zu Asyl, Ausgabe Juni 2017 [Current figures on asylum, issue June 2017]. Retrieved from http://www.bamf.de/SharedDocs/Anlage n/DE/Downloads/Infothek/Statistik/Asyl /aktuelle-zahlen-zu-asyl-juni- 2017.pdf?__blob=publicationFile Bundesamt für Migration und Flüchtlinge (BAMF) (2017b). Verteilung der Asylbewerber in Deutschland nach Geschlecht innerhalb verschiedener Altersgruppen im Jahr 2017 [Distribution of asylum seekers in Germany by gender within different age groups in 2017]. Statista. Retrieved from https://de(.statista.com/statistik/daten/st udie/452165/umfrage/asylbewerber-in- deutschland-nach-geschlecht-innerhalb- altersgruppen/ Bundesinstitut für Berufsbildung (BiBB) (2017). Datenreport zum Berufsbildungsbericht 2017 – Informationen und Analysen zur Entwicklung der beruflichen Bildung. Bonn: BiBB. Bühner, M., Ziegler, M., Krumm, S., & Schmidt- Atzert, L. (2006). Ist der IST 2000 R Rasch-skalierbar?. Diagnostica, 52(3), 119–130. Callahan, R. M., & Humphries, M. H. (2016). Undermatched? School-Based Linguistic Status, College Going and the Immigrant Advantage. American Educational Research Journal, 53(2), 263–295. Castella, H. (2017). The Situation of Refugee Women in Europe, the Spanish State and Catalonia. Diagnosis and Gaps for Improvement (CMC Papers 2017-1). Brussels: Centre Maurits Coppieters (CMC). Cattell, R. B. (1963). Theory of fluid and crystallized intelligence: a critical experiment. Journal of Educational Psychology, 54, 1–22. Chopra, V., & Adelman, E. (2017). The Pursuit, Practicality and Potential of Refugee Education. Zeitschrift für Internationale Bildungsforschung und Entwicklungspädagogik, 40(1), 4–9. Colucci, E., Smidt, H., Devaux, A., Vrasidas, C., Safar Jalani, M., Castaño Munñoz, J. https://www.bamf.de/ http://www.bamf.de/SharedDocs/Anlagen/DE/Downloads/Infothek/Statistik/Asyl/aktuelle-zahlen-zu-asyl-juni-2017.pdf?__blob=publicationFile http://www.bamf.de/SharedDocs/Anlagen/DE/Downloads/Infothek/Statistik/Asyl/aktuelle-zahlen-zu-asyl-juni-2017.pdf?__blob=publicationFile http://www.bamf.de/SharedDocs/Anlagen/DE/Downloads/Infothek/Statistik/Asyl/aktuelle-zahlen-zu-asyl-juni-2017.pdf?__blob=publicationFile http://www.bamf.de/SharedDocs/Anlagen/DE/Downloads/Infothek/Statistik/Asyl/aktuelle-zahlen-zu-asyl-juni-2017.pdf?__blob=publicationFile https://de(.statista.com/statistik/daten/studie/452165/umfrage/asylbewerber-in-deutschland-nach-geschlecht-innerhalb-altersgruppen/ https://de(.statista.com/statistik/daten/studie/452165/umfrage/asylbewerber-in-deutschland-nach-geschlecht-innerhalb-altersgruppen/ https://de(.statista.com/statistik/daten/studie/452165/umfrage/asylbewerber-in-deutschland-nach-geschlecht-innerhalb-altersgruppen/ https://de(.statista.com/statistik/daten/studie/452165/umfrage/asylbewerber-in-deutschland-nach-geschlecht-innerhalb-altersgruppen/ Successful integration of refugee students in higher education 176 (2017). Free Digital Learning Opportunities for Migrants and Refugees. An Analysis of Current Initiatives and Recommendations for their Further Use. Luxembourg: Publications Office of the European Union. Retrieved from http://publications.jrc.ec.europa.eu/repos itory/bitstream/JRC106146/jrc106146.pdf Connor, P. (2010). Explaining the Refugee Gap: Economic Outcomes of Refugees versus Other Immigrants. Journal of Refugee Studies, 23(3), 377–397. Council for Economic Education (CEE) (2010). Voluntary national content standards in economics (2nd ed.). New York: Council for Economic Education. Crea, T. M. (2016). Refugee higher education: Contextual challenges and implications for program design, delivery, and accompaniment. International Journal of Educational Development, 46, 12–22. de Ayala, R. (2009). The Theory and Practice of Item Response Theory. New York/London: The Guilford Press. Demie, F., & Strand, S. (2006). English Language Acquisition and Attainment at the End of Secondary School. Educational Studies, 32(2), 215–231. de Wit, H., & Altbach, P. (2016). The Syrian refugee crisis and higher education. International Higher Education, 84, 9– 10. Dewitte, S., & Lens, W. (2000). Exploring volitional problems in academic procrastinators. International Journal of Educational Research, 33, 733–750. Dielman, T. E. (1989). Pooled cross-sectional and time series data analysis. New York: Dekker. Dochy, F., Segers, M., & Bühl, M. M. (1999). The relation between assessment practices and outcomes of studies: The case of research on prior learning. Review of educational Research, 69(2), 145–186. Dryden-Peterson, S. (2016). Refugee education in countries of first asylum: Breaking open the black box of pre-resettlement experiences. Theory and Research in Education, 14(2), 131–148. Duckworth, A. L., Quinn, P. D., Lynam, D. R., Loeber, R., & Stouthamer-Loeber, M. (2011). Role of test motivation in cognitive functioning testing. Proceedings of the National Academy of Sciences (PNAS), 108(19), 7716–7720. Duffy, B., Smith, K., Terhanian, G., & Bremer, J. (2005). Comparing data from online and face-to-face surveys. International Journal of Market Research, 47(6), 615– 639. Duong, M. T., Badaly, D., Liu, F., Schartz, D., & McCarty, C. A. (2016). Generational Differences in Academic Achievement Among Immigrant Youths: A Meta- Analytic Review. Review of Educational Research, 86(1), 3–41. Educational Testing Service (2008). Reliability and comparability of TOEFL iBT Scores (TOEFL iBT® Research Insight Series No. 3). Princeton, NJ: ETS. Retrieved from https://www.ets.org/s/toefl/pdf/toefl_ibt _research_s1v3.pdf Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists. New Jersey, USA: Lawrence Erlbaum Associates, Publishers. Eurostat (2016). International Standard Classificiation of Education (ISCED). Retrieved from http://ec.europa.eu/eurostat/statistics- explained/index.php/International_Stand ard_Classification_of_Education (ISCED) Eurostat (2017). Migration and migrant population statistics. Retrieved from http://ec.europa.eu/eurostat/statistics- explained/index.php/Migration_and_mig rant_population_statistics Fakeye, D., & Ogunsiji, Y. (2009). English language proficiency as a predictor of academic achievement among ELF students in Nigeria. Journal of Science Research, 37, 490–495. http://publications.jrc.ec.europa.eu/repository/bitstream/JRC106146/jrc106146.pdf http://publications.jrc.ec.europa.eu/repository/bitstream/JRC106146/jrc106146.pdf https://www.ets.org/s/toefl/pdf/toefl_ibt_research_s1v3.pdf https://www.ets.org/s/toefl/pdf/toefl_ibt_research_s1v3.pdf http://ec.europa.eu/eurostat/statistics-explained/index.php/Migration_and_migrant_population_statistics http://ec.europa.eu/eurostat/statistics-explained/index.php/Migration_and_migrant_population_statistics http://ec.europa.eu/eurostat/statistics-explained/index.php/Migration_and_migrant_population_statistics 177 Global Education Review 5(4) Freeman, D. E., & Freeman, Y. S. (2000). Teaching Reading in Multilingual Classrooms. Portsmouth, NH: Heinemann. Furnham, A., Gee, M., & Weis, L. (2016). Knowledge of mental illnesses: Two studies using a new test. Psychiatry Research, 244, 363–369. Griga, D. (2014). Participation in Higher Education of Youths with a Migrant Background in Switzerland. Swiss Journal of Sociology, 40(3), 379–400. Grotjahn, R. (1987). How to Construct and Evaluate a C-Test: A Discussion of Some Problems and Some Statistical Analyses. In. R. Grotjahn, C. Klein-Braley, & D. K. Stevenson (Eds.), Taking their Measure: The Validity and Validation of Language Tests (pp. 219–253). Bochum: Brockmeyer. Haertel, E. H. (1999). Validity arguments for high-stakes testing: In search of the evidence. Educational Measurement: Issues and Practice, 18(4), 5–9. Happ, R., Förster, F., Zlatkin-Troitschanskaia, O., & Carstensen, V. (2016). Assessing the Previous Economic Knowledge of Beginning Students in Germany: Implications for Teaching Economics in Basic Courses. Citizenship, Social and Economics Education, 15(1), 45–57. Harkness, J. A. (2003). Questionnaire translation. In J. A. Harkness, F. J. R. van de Vijver, & P. Ph. Mohler (Eds.), Cross- cultural survey methods (pp. 19–34). Hoboken, NJ: John Wiley & Sons. Harris, V., & Marlowe, J. (2011). Hard Yards and High Hopes: The Educational Challenges of African Refugee University Students in Australia. International Journal of Teaching and Learning in Higher Education, 23(2), 186–196. Harsch, C., & Hartig, J. (2015). Comparing C- tests and Yes/No vocabulary tests as predictors of receptive language skills. Language Testing, 33(4), 555–575. Harsch, C., Tschauschew, J., & Brandt, S. (o.J.). The Kiron English Proficiency Test. Hawkins, J. A., & Filipović, L. (2012). Criterial Features in L2 English. Specifying the Reference Levels of the Common European Framework. Cambridge: Cambridge University Press. Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force Concept Inventory. The Physics Teacher, 30(3), 141–158. Horn, L. J., & Kojaku, L. K. (2001). High School Academic Curriculum and the Persistence Path through College: Persistence and Transfer Behavior of Undergraduates 3 years after Entering 4-Year Institutions. Education Statistics Quarterly, 3(3), 65– 72. Husman, J., & Corno, L. (2010). Volitional Control of Learning. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (3rd ed., pp. 724–731). Oxford: Elsevier. International Test Commission (ITC) (2017). The ITC Guidelines for Translating and Adapting Tests (2nd ed.). International Journal of Testing, 18(2), 101–134. Jang, K., Hahn, K., & Kim, K. (2010). Comparative Korean Results of TUCE with U.S. and Japan. In M. Yamaoka, W. Wallstad, M. Watts, T. Asano, & S. Abe (Eds.), Comparative Studies on Economic Education in Asia-Pacific Region (pp. 53– 78). Tokyo: Shumpusha Publishing. Joyce, A., Earnest, J., De Mori, G., & Silvagni, G. (2010). The Experiences of Students from Refugee Backgrounds at Universities in Australia: Reflections on the Social, Emotional and Practical Challenges. Journal of Refugee Studies, 23(1), 82–97. Jungblut, J., & Pietkiewicz, K. (2017). Refugees Welcome? Recognition of qualifications held by refugees and their access to higher education in Europe – country analyses. Brüssel: European Students’ Union (ESU). Retrieved from Successful integration of refugee students in higher education 178 Kaplan, I., Stolk, Y., Valibhoy, M., Tucker, A., & Baker, J. (2016). Cognitive assessment of refugee children: Effects of trauma and new language acquisition. Trancultural Psychiatry, 53(1), 81–109. Kira, I. A., Lewandowski, L., Yoon, J., Somers, C., & Chiodo, L. (2012). The Linear and Nonlinear Associations between Multiple Types of Trauma and IQ Discrepancy Indexes in African American and Iraqi Refugee Adolescents. Journal of Child & Adolescent Trauma, 5(1), 47–62. Krapp, A. (1999). Interest, motivation and learning: An educational-psychological perspective. European journal of psychology of education, 14(1), 23–40. Krashen, S., & McField, G. (2006). What works? Reviewing the latest evidence on bilingual education. Language Learner, 1(2), 7–10. Kuh, G. D., Kinzie, J., Buckley, J. A., Bridges, B. K., & Hayek, J. C. (2007). Piecing Together the Student Success Puzzle: Research, propositions, and recommendations. ASHE Higher Education Report, 32(5), 1–182. Larsen, M. S., Kornbeck, K. P., Kristensen, R. M., Larsen, M. R., & Sommersel, H. B. (2013). Dropout Phenomena at Universities: What is Dropout? Why does Dropout Occur? What Can be Done by the Universities to Prevent or Reduce it? A systematic review. Copenhagen: Danish Clearinghouse for Educational Research, Department of Education, Aarhus University. Liepmann, D., Beauducel, A., Brocke, B., & Amthauer, R. (2007). Intelligenz- Struktur-Test 2000 R [Intelligence Structure Test 2000 R]. Göttingen: Hogrefe. Little, D. (2007). The Common European Framework of Reference for Languages: Perspectives on the Making of Supranational Language Education Policy. The Modern Language Journal, 91(4), 645–655. Lorisika, I., Cremonini L., & Safar Jalani, M. (2015). Study to Design a Programme / Clearinghouse Providing Access to Higher Education for Syrian Refugees and Internal Displaced Persons. Retrieved from https://ris.utwente.nl/ws/files/5138553/F inal%20Report%20- %20Study%20Higher%20Education%20S yrian%20refugees%20and%20IDP.pdf McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, N.J.: Lawrence Erlbaum Associates. Mendenhall, M. A., Russell, S. G., & Bruckner, E. (2017). Urban Refugee Education: Strengthening Policies and Practices for Access, Quality, and Inclusion. Columbia: Columbia University Academic Commons. Retrieved from https://doi.org/10.7916/D8R49WGP Mislevy, R. J., & Haertel, G. D. (2006). Implications of Evidence-Centered Design for Educational Testing. Education Measurement: Issues and practice, 25(4), 6–20. Moll, L. C. (1992). Bilingual Classroom Studies and Community Analysis: Some Recent Trends. Educational Researcher, 21(2), 20–24. Morrice, L. (2013). Refugees in Higher Education: Boundaries of belonging and recognition, stigma and exclusion. International Journal of Lifelong Education, 32(5), 652–668. Niemi, R. G., Craig, S. C., & Mattei, F. (1991). Measuring internal political efficacy in the 1988 National Election Study. The American Political Science Review, 85, 1407–1413. Nieminen, P. (2013). Representational consistency and the learning of forces in upper secondary school physics. Jyväskylä: University of Jyväskylä. Nieminen, P., Savinainen, A., & Virri, J. (2010). Force Concept Inventory-based multiple- choice test for investigating students’ https://ris.utwente.nl/ws/files/5138553/Final%20Report%20-%20Study%20Higher%20Education%20Syrian%20refugees%20and%20IDP.pdf https://ris.utwente.nl/ws/files/5138553/Final%20Report%20-%20Study%20Higher%20Education%20Syrian%20refugees%20and%20IDP.pdf https://ris.utwente.nl/ws/files/5138553/Final%20Report%20-%20Study%20Higher%20Education%20Syrian%20refugees%20and%20IDP.pdf https://ris.utwente.nl/ws/files/5138553/Final%20Report%20-%20Study%20Higher%20Education%20Syrian%20refugees%20and%20IDP.pdf https://doi.org/10.7916/D8R49WGP 179 Global Education Review 5(4) representational consistency. Physical Review Special Topics - Physics Education Research, 6(2), 1–12. Norris, J. M. (2006). Development and evaluation of a curriculum-based German C-test for placement purposes. In R. Grotjahn (Ed.), The C-Test: Theory, empirical research, applications (pp. 45– 83). Frankfurt am Main: Lang. North, B. (2014). Putting the Common European Framework of Reference to good use. Language Teaching, 47(2), 228–249. Pagel, L., Richter, D., & Schupp, J. (2018). Kognitive Potenziale von Geflüchteten [Cognitive potentials of refugees]. In H. Brücker, N. Rother & J. Schupp (Eds.), IAB-BAMF-SOEP-Befragung von Geflüchteten 2016: Studiendesign, Feldergebnisse sowie Analysen zu schulischer wie beruflicher Qualifikation, Sprachkenntnissen sowie kognitiven Potenzialen [IAB-BAMF-SOEP Refugee Survey 2016: study design, field results and analyses of school and vocational qualifications, language knowledge and cognitive potential] (pp. 54–67). Berlin: DIW Berlin. Pellegrino, J. W. (2016). Introduction to Special Section of Educational Psychologist on Educational Assessment: Validity Arguments and Evidence—Blending Cognitive, Instructional, and Measurement Models and Methods. Educational Psychologist, 51(1), 57–58. Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). The nature of assessment and reasoning from evidence. In J. W. Pellegrino, N. Chudowsky, & R. Glaser (Eds.), Knowing what students know: The science and design of educational assessment (pp. 37–54). Washington, DC: National Academy Press. Rampelt, F., & Suter, R. (2017). Recognition of prior learning – outcome-oriented approaches to the recognition and assessment of MOOC-based digital learning scenarios. In L. Gómez Chova, A. López Martínez, & I. Candel Torres (Eds.), EduLearn 2017. 9th Annual International Conference on Education and New Learning Technologies (pp. 6645–6653). Barcelona, Spain. Reinhardt, F., Zlatkin-Troitschanskaia, O., Deribo, T., Happ, R., & Nell-Müller, S. (2018). Integrating refugees into higher education – the impact of a new online education program for policies and practices. Policy Reviews in Higher Education, 2(2), 198–226. Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the GLB: comments on Sijtsma. Psychometrika, 74, 145–154. Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students' academic performance: a systematic review and meta-analysis. Psychological Bulletin, 138(2), 353–387. Rienties, B., Beausaert, S., Grohnert, T., Niemantsverdriet, S., & Kommers, P. (2012). Understanding academic performance of international students: the role of ethnicity, academic and social integration. Higher Education, 63(6), 685–700. Rios, J. A., & Liu, O. L. (2017). Online Proctored Versus Unproctored Low-Stakes Internet Test Administration: Is There Differential Test-Taking Behavior and Performance?. American Journal of Distance Education, 31(1), 1–14. Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130(2), 261–288. OECD, Eurostat, & UNESCO Institute for Statistics (2015). ISCED 2011 Operational Manual: Guidelines for Classifying National Education Programmes and Related Qualifications. Paris: OECD. Successful integration of refugee students in higher education 180 Schiefele, U., Krapp, A., Wild, K. -P., & Winteler, A. (1993). Der "Fragebogen zum Studieninteresse" (FSI). Diagnostica, 39, 335–351. Schmidgall, J. E., & Powers, D. E. (2017). Technology and High‐stakes Language Testing. In C. A. Chapelle & S. Sauro (Eds.), The Handbook of Technology and Second Language Teaching and Learning (pp. 317–331). Dublin: Wiley Blackwell. Schroeder, J., & Seukwa, L. H. (2017). Access to Education in Germany. In A. Korntheuer, P. Pritchard, D. B. Maehler, & GESIS - Leibniz-Institut für Sozialwissenschaften (Eds.), Structural Context of Refugee Integration in Canada and Germany (Vol. 15) (pp. 59–65). Köln: GESIS. Retrieved from http://nbn- resolving.de/urn:nbn:de:0168-ssoar- 52101-0 Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107–120. Sijtsma, K., & Vvan der Ark, L. A. (2015). Conceptions of reliability revisited and practical recommendations. Nursing Research, 64, 128–136. Stenlund, T., Lyrén, P. E., & Eklöf, H. (2017). The successful test taker: exploring test- taking behavior profiles through cluster analysis. European Journal of Psychology of Education, 33(2), 403–417. Suter, R., Harsch, C., & Brandt, S. (2017). Entwicklung eines Englisch- Einstufungstests für Geflüchtete [Development of an English placement test for refugees]. Presented at the annual conference of the Gesellschaft für Empirische Bildungsforschung [Society for Empirical Educational Research], Heidelberg. Retrieved from http://www.gebf2017.de/aktuelles/abstra cts_Einzelbeitraege_GEBF- 2017_030317.pdf Swope, K. J., & Schmitt, P. M. (2006). The Performance of Economics Graduates over the Entire Curriculum: The Determinants of Success. Journal of Economic Education, 37(4), 387–394. Thurstone, L. L. (1938). Primary mental abilities. Chicago: University of Chicago Press. Tjaden, J. D., & Hunkler, C. (2017). The optimism trap: Migrants’ educational choices in stratified education systems. Social Science Research, 67, 213–228. Tohidi, H., & Jabbari, M. M. (2012). The effects of motivation in education. Procedia- Social and Behavioral Sciences, 31, 820– 824. United Nations Educational, Scientific and Cultural Organization (UNESCO) (2012). International Standard Classification of Education ISCED 2011. Montreal, Quebec: UNESCO Institute for Statistics. United Nations Educational, Scientific and Cultural Organization (UNESCO) (2017). Syrian Arab Republic. Retrieved from http://uis.unesco.org/en/country/sy United Nations High Commissioner for Refugees (UNHCR) (2017). UNHCR- Bericht: Flucht und Vertreibung erreichen 2016 neuen Höchststand. Retrieved from http://www.unhcr.org/dach/de/15212- globaltrends2016.html van Herpen, S. G. A., Meeuwisse, M., Hofman, W. H. A., Severiens, S. E., & Arends, L. R. (2017). Early predictors of first-year academic success at university: pre- university effort, pre-university self- efficacy, and pre-university reasons for attending university. Educational Research and Evaluation, 23(1-2), 52–72. Verhelst, N., van Avermaet, P., Takala, S., Figueras, N., & North, B. (2009). Common European Framework of Reference for Languages: learning, teaching, assessment. Cambridge: Cambridge University Press. Wagener, D. (2003). Der Computerwissenstest CWIS-4: Befunde zur Reliabilität und Validität [Computer Science test CWIS-4. http://nbn-resolving.de/urn:nbn:de:0168-ssoar-52101-0 http://nbn-resolving.de/urn:nbn:de:0168-ssoar-52101-0 http://nbn-resolving.de/urn:nbn:de:0168-ssoar-52101-0 http://www.gebf2017.de/aktuelles/abstracts_Einzelbeitraege_GEBF-2017_030317.pdf http://www.gebf2017.de/aktuelles/abstracts_Einzelbeitraege_GEBF-2017_030317.pdf http://www.gebf2017.de/aktuelles/abstracts_Einzelbeitraege_GEBF-2017_030317.pdf http://uis.unesco.org/en/country/sy http://www.unhcr.org/dach/de/15212-globaltrends2016.html http://www.unhcr.org/dach/de/15212-globaltrends2016.html 181 Global Education Review 5(4) Results for reliability and validity]. Zeitschrift für Personalpsychologie, 2(4), 169–181. Walstad, W. B., Rebeck, K., & Butters, R. B. (2013a). Test of economic literacy: Examiner’s manual (4th ed.). New York: Council for Economic Education. Walstad, W. B., Rebeck, K., & Butters, R. B. (2013b). The Test of Economic Literacy: Development and Results. Journal of Economic Education, 44(3), 298–309. Wright, L.-A., & Plasterer, R. (2012). Beyond Basic Education: Exploring Opportunities for Higher Learning in Kenyan Refugee Camps. Refuge: Canada's Journal on Refugees, 27(2), 42–56. Yamaoka, M., Asano, T., & Abe, S. (2010). The Present State of Economic Education in Japan. Journal of Economic Education, 41(4), 448–460. Zentralarchiv für empirische Sozialforschung (ZA), & Zentrum für Umfragen, Methoden und Analysen (ZUMA) e.V. (2014). Politikinteresse. Zusammenstellung sozialwissenschaftlicher Items und Skalen. doi:10.6102/zis181 Zeus, B. (2011). Exploring Barriers to Higher Education in Protracted Refugee Situations: The Case of Burmese Refugees in Thailand. Journal of Refugee Studies, 24(2), 256–276. Zheng, S., Rosson, M. B., Shih, P. C., & Carroll, J. M. (2015). Understanding Student Motivation, Behaviors and Perceptions in MOOCs. Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (pp. 1882–1895). ACM. doi:10.1145/2675133.2675217 Zorlu, A. (2013). Ethnic disparities in higher education. IZA Journal of Migration, 2(3), 1–21. About the Author(s) Dr. Olga Zlatkin-Troitschanskaia is a professor and Chair of Business and Economics Education at Johannes Gutenberg University, Mainz, Germany. Dr. Roland Happ is a research associate and lecturer at the Department of Business and Economics Education at Johannes Gutenberg University, Mainz, Germany. Sarah Nell-Müller is a research associate at the Department of Business and Economics Education at Johannes Gutenberg University, Mainz, Germany. Tobias Deribo is a research associate at the Department of Business and Economics Education at Johannes Gutenberg University, Mainz, Germany. Franzisca Reinhardt is a research associate and lecturer at the Department of Business and Economics Education at Johannes Gutenberg University, Mainz, Germany. Miriam Toepper is research associate at the Department of Business and Economics Education at Johannes Gutenberg University, Mainz, Germany. https://www.dict.cc/englisch-deutsch/reliability.html https://www.dict.cc/englisch-deutsch/validity.html