Iberica 13 IBÉRICA 13 [2007]: 147-170 Abstract This paper analyzes the reasons why in a group of 85 engineering students, some were not able to pass our English for Specific Purposes courses (ESP Surveys One and Two). They took two tests, Grammar Surveys One and Two, and completed a questionnaire, prior to beginning of the courses. The tests focused on grammar points of increasing complexity, and the questionnaire gathered information about factors influencing learner differences. ANOVA analyses showed a significant positive effect of the Grammar Surveys on the ESP Surveys. Amongst the items included in the questionnaire, only one variable, “previous academic performance” (PAP), showed up as having a significant positive effect on the ESP Surveys. It seems then, that the marks obtained in these ESP Surveys depend exclusively upon the results obtained in the Grammar Surveys, and upon the PAP of the students. The findings are discussed in terms of our degree of responsibility in the learning process of our students, and on predictable performance patterns. Key words: ESP courses, learning process, adult language learners, performance predictability. Resumen ¿Est�n mis alumnos de ingenier�a ling��sticamente preparados para recibir mis cursos de IFE? Este artículo analiza las razones por las que en un grupo de 85 estudiantes de ingeniería, algunos no fueron capaces de aprobar nuestros cursos de Inglés para Fines Específicos (ESP Survey One y ESP Survey Two). Antes de comenzar los How linguistically ready are my engineering students to take my ESP courses? Joseba M. González Ardeo Universidad del País Vasco / Euskal Herriko Unibertsitatea fipgoarj@lg.ehu.es 147 08 JOSEBA.qxp 6/4/07 09:07 Página 147 cursos, realizaron dos pruebas, Grammar Survey One y Grammar Survey Two, y cumplimentaron un cuestionario. Las pruebas se centraban en puntos de gramática de complejidad creciente, y el cuestionario recogía información sobre factores que influyen en las diferencias entre estudiantes. El análisis de varianza (ANOVA) mostró un efecto significativo positivo de los Grammar Surveys en los ESP Surveys. Entre los ítems incluidos en el cuestionario, sólo una variable, previous academic performance (resultados académicos previos), demostró tener un efecto significativo positivo en los ESP Surveys. Parece entonces que las notas obtenidas en estos ESP Surveys dependen exclusivamente de los resultados obtenidos en los Grammar Surveys y en los resultados académicos previos de los estudiantes. Los resultados obtenidos se tratan a nivel de nuestra responsabilidad en el proceso de aprendizaje de nuestros alumnos, y en patrones predecibles de resultados. Palabras clave: cursos de IFE, proceso de aprendizaje, estudiantes adultos de lenguas, predicción de resultados académicos. Introduction When we were students, some of our classmates were better language learners, objectively measured, than others even though we all were exposed to similar teaching, used the same learning material and had similar opportunities to practice English. However, teachers know that not all learners behave in exactly the same way. While some students always adopt a very active role, others prefer to remain neutral or even passive towards learning. There are some students who progress very fast, apparently with little effort, whereas others put a lot of work into learning and they obtain poor results. There are also students who prefer to learn things by heart while others opt for learning through practice. Therefore, it stands to reason that the existence of learners with different capacities and abilities seems to be a fact well worth researching. Factors influencing learner differences when learning a language are usually grouped under headings such as “cognitive” (intelligence, language learning aptitude, cognitive style, learning strategies), “affective” (attitudes towards language learning, motivation for language learning), and “physical and psychological” (age, gender, personality), but other taxonomies are also possible, for instance, “individual variables” (intelligence, linguistic aptitude, personality traits, cognitive style), “socio-structural variables” (age, gender, socio-cultural level, social setting), “psycho-social variables” (attitude, motivation), and “psycho-educational variables” (L2 learning context). All JOSEBA M. GONZÁLEZ ARDEO 148 IBÉRICA 13 [2007]: 147-170 08 JOSEBA.qxp 6/4/07 09:07 Página 148 these variables have been often analyzed both for children and for adults (Wenden, 1986; Oxford & Nyikos, 1989; Ehrman & Oxford, 1990). Moreover, Gardner (1985) developed a model of education to explain the relationship between certain factors and the learning of a second language, thus showing that the variables under all those aforementioned headings should not be regarded as mutually exclusive or independent since quite often the effects of some interact with others. In any educational program at the university level, both the students and the lecturers are individuals whose performances can be evaluated. Any independent observer would agree with the statement “The most frequently evaluated individuals are usually the students”. However, that part of the labor force from a university not engaged in administrative duties, but in lecturing ones, is well aware that there are many different mechanisms to evaluate its performance which, by the way, affects careers not only from a professional point of view, but also from an economical one. But, who decides whether a student should or should not take the course a member of the teaching staff has designed or is about to design? In many programs, entrance is nearly an automatic process since no selection or entrance test is required. In other words, prerequisites of knowledge or skills are not necessarily connected to those needed in the learning process of the target language but to factors such as “branch of engineering the student is enrolled in”, “number of credits necessary for completing his/her degree”, and so on. This is then the background used to plan a research study where the students’ readiness for taking specialized ESP courses was one of the variables evaluated, together with certain performance-related indicators. The term “readiness” is not used arbitrarily, but quite deliberately since it takes into account the extent to which the students were potentially ready to take the ESP courses offered within a particular institution. Tests to filter out candidates who do not yet display readiness In many parts of the world, nation-wide university entrance examinations include a section where knowledge of a foreign language, mainly English, is assessed in addition to assessments of academic subjects such as mathematics, chemistry, history or art. Nevertheless, this pre-selection HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 149 08 JOSEBA.qxp 6/4/07 09:07 Página 149 process does not always guarantee in itself linguistic homogeneity since the minimum level required to pass it is rather easily attained. One of the tasks of lecturers is to carry out achievement tests and/or formal assessments which usually come at the end of a long period of learning. Their main purposes are, on the one hand and from the lecturer’s point of view, to show who will be prepared to cope best in the target situation and, on the other hand and from the learners’ point of view, to provide the key to promotion to a more advanced course or simply to fulfill a “pass” requirement in order to complete the different courses engineering studies are divided into. But, what happens with those individuals who do not pass these assessments? What is the lecturer’s degree of responsibility for their failure? To what extent are other agents responsible? Were they ready, linguistically speaking, to take the course they were offered? Engineering students in ESP courses are usually grouped according to factors such as expected level of language ability, expected language learning aptitude that, at least in theory, should bring homogeneity to the groups, and this together with a more objective factor, that is, the branch of engineering the student is enrolled in. This amalgam of factors will not necessarily help lecturers predict future performances of these students. Brown et al. (1994) remind us that as providers of education it is important to step back and consider why teachers assess. Among the reasons why assessment is useful, the most frequently mentioned are “motivation”, “creating learning activities”, “feedback to the student (identifying strengths and weaknesses)”, “feedback to the staff on how well the message is getting across”, and “to judge performance (grade/degree classification)”. However, “quality assurance (internal and external to the institution)” has been recently included in this set. Regarding grouping, the most rational criterion appears to be based on language ability. For this reason, it was our purpose to measure students’ language abilities not only after taking our courses but also before taking them, with a clear purpose in mind, namely, to know how ready our engineering students are for ESP instruction. An objective way of tackling this issue seems to be concentrated in the word “assessment”, but new questions arise. How and what should be assessed? Keeping in mind the above questions, a questionnaire and two tests were designed. The former included items to gather as much information as possible regarding the linguistic background of the students. The latter were JOSEBA M. GONZÁLEZ ARDEO 150 IBÉRICA 13 [2007]: 147-170 08 JOSEBA.qxp 6/4/07 09:07 Página 150 connected to the learning objectives of the syllabus the students were about to take. Thus, the following hypotheses were put forward: Main hypotheses: 1.1. Students will exhibit appreciable differences when tested on specific language learning, depending upon their previous academic performance within the field of English for General Purposes. 1.2. Grammar Surveys carried out prior to the ESP courses will tell us in advance which students would pass or fail the ESP tests the first time around. Secondary hypotheses: 2.1. Variables such as age, gender, mother tongue, personality traits, social setting and educational level, will not affect performance in the ESP courses administered. 2.2. Variables such as language background, attitude towards ESP, motivation, former teachers’ efficiency, language learning aptitude, previous academic performance, communicative aptitude, and phonetic coding ability will affect performance in the ESP courses and their corresponding assessments. Main hypothesis 1.1 posits a correlation between the degree of linguistic proficiency in the General English of the students and their chances of success in the ESP courses. Hypothesis 1.2 predicts that there will be performance differences in the ESP tests, between those who pass and those who do not pass certain grammar surveys used as filters. Secondary hypothesis 2.1 considers a negative correlation between some variables (age, gender, mother tongue, personality traits, social setting and educational level) and the learners’ performances in the ESP tests. Hypothesis 2.2 assigns better expected results in the ESP tests to learners with a rich linguistic background, a positive motivation and attitude towards ESP, and a positive feeling about the students’ former teachers’ efficiency, their own language and communicative aptitude, and their phonetic coding ability. HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 151 08 JOSEBA.qxp 6/4/07 09:07 Página 151 Participants The study was completed in the Basque Country. The participants were 85 engineering students from the Industrial Technical Engineering College in Bilbao (ITEC-B) in the 19-24 age range. Their performances were evaluated before and after taking two short specialized ESP courses. They belonged to the four different branches currently studied within the industrial section, namely, Mechanical Engineering (30 students), Electrical Engineering (15), Industrial Electronic Engineering (27), and Industrial Chemical Engineering (13). Most students enter the College via two local options: 1) high school; 2) vocational training. The former follows a much more theoretical approach than the latter but, in the case of English as a subject matter, grammar is the main focus of attention in both groups. Original variables and research instruments Independent variables These were measured by means of a self-report questionnaire (see Appendix I) that provided raw data on the learners’ physical and psychological factors (age, gender, and personality trait), affective factors (attitude towards ESP, motivation), cognitive factors (language learning aptitude, communicative aptitude, phonetic coding ability), and socio-educational factors (mother tongue, social setting, educational level, language background, former teachers’ efficiency, and previous academic performance). Moreover, an item on the use of English as a means of instruction in subjects other than English was included in the questionnaire. The age variable can be easily measured, and this is not the case with other important SLA factors. It has been generally taken for granted that young learners learn languages more easily than adults but research seems to indicate that this is only the case when it comes to pronunciation. Gender is a variable traditionally related to the linguistic development of L1 and L2. In both cases, two types of studies have been carried out: quantitative and qualitative. In the former, the differences between linguistic developments of males and females are merely stated. In the latter, a more recent one, results demonstrate convincingly that the language reflects social JOSEBA M. GONZÁLEZ ARDEO 152 IBÉRICA 13 [2007]: 147-170 08 JOSEBA.qxp 6/4/07 09:07 Página 152 structures and power hierarchies between sexes. Gender cannot be considered an isolated variable since, related to other variables (e.g. race, age, social status), it reflects the social hierarchical structure (Giles & Robinson, 1990). When teenagers are evaluated most studies agree (Burstall, 1975), and they state that girls obtain better results than boys. However, when adult students’ results are compared, in most studies (Liski & Putanen, 1983), no differences are observed. As far as the students’ mother tongue is concerned, it is generally believed that bilingual students acquiring a third language outperform monolingual ones. Many studies show that bilingualism has a positive effect upon L3 acquisition (Albert & Obler, 1978; Clark, 1987). However, other studies show that bilinguals do not present any real advantage over monolinguals (González, 2004). Ringbom (1985) suggests that bilingualism has positive effects upon L3 acquisition when the languages are learnt within the same context. This is the case for most bilingual students in the Basque Country. All native Basque speakers speak Spanish, although the opposite is not true. The item is pertinent to the topic under discussion, when dealing with monolinguals and bilinguals, since the students may present differences when comparing L2 acquisition and L3 acquisition (Cenoz & Genesse, 1998; González, 2004). Teachers could foster certain attitudes towards language learning that could have direct and/or indirect influences on the students’ personality profile. Confidence in the teacher is vital, and s/he should create the best possible atmosphere in the classroom so that learners can overcome any anxiety they may feel since we learn best when we are relaxed. However, despite any effort made in that direction, the student’s involvement in the teaching and learning tasks is a basic ingredient for success in language learning. In other words, extroversion, tolerance of ambiguity, low anxiety, a disposition to take risks and ‘anomie’ seem to indirectly correlate in most cases with language success. Given these results, personality becomes an important factor in the acquisition of communicative skills but the connection is not so close in terms of pure linguistic ability. In fact, Lightbown and Spada (1993: 39) state that “it is probably not personality alone, but the way in which it combines with other factors that contributes to second language learning”. Social setting is one of the classical variables included when acquisition and performance are evaluated. Social groups develop communicative ways that facilitate cooperation and coexistence among their members (Giles & Robinson, 1990). In this way, communicative practice reflects social power HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 153 08 JOSEBA.qxp 6/4/07 09:07 Página 153 from the members of a group. The differences become social deficits when the groups are different and the standards of one of the groups dominate. This deficit is due to the comparison of groups from various statuses and, without comparison, no disadvantage would exist (Giles & Robinson, 1990). The way the educational level variable has been considered in this research is somewhat exceptional since this is a peculiarity of the students from our College. Our two main student recruitment sources come from Vocational Training and Higher Certificate. The language background variable was included to check not only the extent the groups of students differ from each other, but also to filter the motivation variable since the fact of taking voluntary English courses could be considered as connected to motivation. If motivation exists, success in language learning is almost guaranteed (Ellis, 1985; Gardner, 1985). According to Gardner (1985), the different components of motivation are effort+setting and desire to achieve goals+attitudes. Effort is the first element in motivation and, according to Gardner, it may be triggered by several factors such as social pressures, a great achievement need, etc. Setting and desire to achieve goals is the component that serves to channel the effort. Finally, motivation will vary depending on the different attitudes individuals possess toward the learning of the language. Then, these two affective factors, attitude and motivation, should somehow be represented in this study since motivation plays a key role in SLA. Besides, learning environments or communities also shape attitudes and motivation toward the learning of the target language (González, 2003). The reason why an item on “Former teachers’ efficiency” was included in the questionnaire can be justified if the teachers’ responsibility to motivate is considered. Teachers’ role in motivating students appears to be closely related to success in language learning, at least when in primary and secondary school. At university, the lecturer’s function is closer to that of “facilitator” or “provider of materials” as the learners must take over more responsibility for their own learning, thus becoming more autonomous. Then, our expectations in connection to this question are related to the past, and they consist of knowing the extent to which students were motivated by their English teachers. Aptitude is the natural ability people have to learn languages and all of us possess it to varying degrees. Researchers are interested in aptitude, a natural JOSEBA M. GONZÁLEZ ARDEO 154 IBÉRICA 13 [2007]: 147-170 08 JOSEBA.qxp 6/4/07 09:07 Página 154 aptitude for languages, as a predictor of performance. Skehan (1989: 38) believes that “aptitude is consistently the most successful predictor of language learning success”. The student’s performance when learning English for General Purposes is named Previous Academic Performance in the questionnaire. This factor is not usually included in taxonomies on individual factors in a learner’s development but linguistic background is a priori connected to future linguistic performance, at least, when differences between secondary school and university are evaluated. Communicative aptitude is another important factor in SLA, and new versions of the already existing aptitude tests should be devised with the purpose of measuring not only grammatical, memory and analytical language abilities but also the learner’s capacity to communicate meaning. Current definitions of communicative competence or communicative language ability no longer talk of the dichotomy between competence and performance, but lecturers are usually aware of the existence of learners with different capacities and abilities in their classrooms, that is, provision should be made for both “strong” and “weak” learners. Nevertheless, this is not easy at the university since in most cases the same information is provided to all students. Phonemic or phonetic coding ability –the ability to discriminate and recall new sounds– is one of the four main components in language aptitude in Carroll & Sapon’s (1959) Modern Language Aptitude Test. Although the correlation between this factor and the ability to communicate meaning may not be clear, it has been included because important differences amongst students at the ITEC-B are usually present. Why certain items were included in the questionnaire but not others is just a matter of choice. Matters do not seem to be fully clarified with respect to intelligence, an item that apparently should have been included in the questionnaire, from the point of view of its relationship with SLA. In fact, some researchers (Genesse, 1976) seem to indicate that intelligence may influence the acquisition of some skills associated with SLA, particularly those used in the formal study of an L2, but others (Lightbown & Spada, 1993) consider that no correlation between intelligence and L2 communicative learning exists. HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 155 08 JOSEBA.qxp 6/4/07 09:07 Página 155 Dependent variables These were measured via different kinds of tests, two Grammar Surveys (a multiple-choice test and a cloze test), and the official ESP examinations administered in the ITEC-B, ESP Survey One and ESP Survey Two. On the other hand, three instruments were employed to collect data for this study: a questionnaire on factors influencing learner’s development, two tests on English grammar and usage (before the ESP courses), and two official examinations (after the ESP courses). 1) Questionnaire The first instrument consisted of the aforementioned questionnaire (see Appendix I) that students were invited to complete. The items selected pursue a priori a clear goal, namely, to see to what extent there is a correlation between the differences of individuals in the previously mentioned factors, and their performances. 2) Readiness tests Pre-university students are supposed to be trained in grammar and usage but year in, year out, dramatic differences among individuals belonging to the same group are observed. If the expected level of grammar and usage guarantees, to some extent, correct understanding of the language used in the ESP courses, then, some kind of test based on grammar and usage would shed new light on the causes of poor marks attained by certain students in the official examinations. The importance of grammar in SLA has been highlighted by a considerable body of research (Doughty & Williams, 1998; Purpura, 2004) showing that language learning is enhanced when grammar instruction is both form and meaning-based and when the development of a learner’s explicit and implicit knowledge of grammar is emphasized. One of the major uses of language tests mentioned by Bachman (1990: 54) was taken into account, that is, “indicators of abilities or attributes that are of interest in research on language, language acquisition, and language testing”. The first test administered “readiness test 1 (Grammar Survey One –a multiple-choice test–)” was adapted from those tests designed by Swan & Walter (1997) and, in much the same way, it was divided into three main sections. The first one focused on basic grammar points: determiners (articles, possessives and demonstratives, etc.); pronouns and nouns; JOSEBA M. GONZÁLEZ ARDEO 156 IBÉRICA 13 [2007]: 147-170 08 JOSEBA.qxp 6/4/07 09:07 Página 156 comparatives and superlatives; be, do and have; modal auxiliary verbs (basic rules); present tenses; and basic sentence-structures; prepositions (in, at, to); etc. The second referred to intermediate grammar points: some and any; much, many, etc.; adjectives and adverbs; word order (adverbs with the verb); like/as; have (got); modal auxiliary verbs; future tenses; perfect and past tenses; passives; infinitives; preposition + -ing, indirect speech; structures with if, unless, in case, etc.; relative who, which and that; etc. Eventually, the last section included advanced grammar points such as noun + noun; order of adjectives before nouns; modal auxiliary verbs (special problems); future tenses after if and when; tenses with since and for; passives; to …ing; negative sentence structures; dropping sentence beginnings; non-identifying relative clauses; opposite vs. in front of; etc. By using this unbiased-for-all- these-students source of information, it could be guaranteed that the answers to the items were neither “canned” nor rehearsed. The multiple-choice test included ninety items (30 items per section) but, following the above mentioned line of work (Swan & Walter, 1997), they could contain one or more correct answers (a degree of difficulty an adult university student should a priori be able to overcome). In fact, the ninety items contained 116 correct answers out of 299 potential choices. Moreover, the sections were clearly separated (one sheet of paper per section) and were entitled “basic”, “intermediate”, and “advanced” respectively. Everybody had to fill in test number one before completing test number two and lastly test number three. Another point to be mentioned here is that all correct answers were given a “+1” mark (e.g. an item with two correct answers was given a “+2” mark), while wrong answers were given a “-0.25” mark. This strategy was followed with a clear intention, on the one hand to penalize errors –the penalty was chosen approximately and using the logic of arithmetic– and, on the other, to minimize “guesstimates”. Moreover, the research carried out is not invalidated by the penalizations due to the way the results were used. These test formats are far from reflecting authenticity of language or the placement of language in context (real-world speech does not extend to multiple choice conversations!), but subjective marking, and the associated problems of reliability, were overcome by using them. Besides, all these adult students are familiarized with this type of task. In order to somewhat crosscheck those problems, a complementary readiness test (Grammar Survey Two –a cloze test) was also administered to HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 157 08 JOSEBA.qxp 6/4/07 09:07 Página 157 the students. Thirty words –any part of speech– were randomly removed from a text (this task is both objective and integrative, and one can draw on a wider range of language abilities). The topic chosen for the text was selected after carefully considering the situation. “A career in engineering” was a text that described jobs and degrees of responsibility an engineer might expect to face. The criterion followed when marking this test was the same as the one mentioned above, that is, a “+1” mark for correct answers and a “-0.25” mark for wrong ones. Only a restricted range of skills in language ability was measured and evaluated. Skills like reading, writing, listening, speaking, pragmatics, sociolinguistic competence, etc. were not evaluated due to local features –they seldom are part of a formal assessment in secondary school. Cloze procedure has often been used in well-known test batteries (Brown, 1978). Cloze tests are regarded as objective testing techniques, and also integrative (Brown, 1983), since a large number of items are tested, full linguistic, semantic and stylistic context is provided for each item, the technique operates beyond the sentence level, and the learner has to draw on a wide range of language sub-skills in order to complete the test successfully. A quick look at the principles of language testing shows the tension that exists in language testing in general, since, for example, the more reliable a test is, the less valid it is likely to be. Cloze tests sacrifice validity in favor of reliability and they focus on receptive skills rather than on productive ones, mainly because they try to be a backward-looking assessment to see to what degree the usage-based syllabuses have been assimilated by the learner. The cloze test from Grammar Survey Two is a norm-referenced test. A passing grade tells you very little about what the learner can actually do with the language but the combination of both tests (multiple choice + cloze test) fulfills some characteristics which are generally thought to be desirable for a formal test: 1) they have high utility since a lot of feedback to assist in the planning of these ESP courses is given; 2) they discriminate between stronger and weaker students; 3) their practicality –in terms of equipment required, and time to set, administer, and mark– was high; 4) they are reliable because of the scoring, but lack validity as a result of not assessing all the skills. In order to enhance test validity and fairness, potential sources of unfairness in testing have been eliminated or, at least, minimized. These include items such as external pressures, availability of information about the tests, the JOSEBA M. GONZÁLEZ ARDEO 158 IBÉRICA 13 [2007]: 147-170 08 JOSEBA.qxp 6/4/07 09:07 Página 158 assessment tools themselves, and possible unfairness due to the most vulnerable element in the testing process, the human marker (Kunnan, 2000). The students were also divided into two subgroups per branch. The former, Student Subgroup One, included individuals with a potentially favorable enough attitude towards English in general based on the results obtained in Grammar Survey One and Grammar Survey Two, and the latter, Student Subgroup Two, included the rest. Maximum potential marks in both tests were 116 and 30 respectively (overall base = 146). Then, our previous “good-enough” consideration means that all those students who got 60 or more marks were, at least potentially, good enough to take the courses (60 out of 146: approximately 41%), that is, Student Subgroup One (S1) ≥ 60 and Student Subgroup Two (S2) < 60. These cut-off scores were somewhat arbitrarily chosen but with a clear idea in mind, to sort out the groups. In our agenda, S2 would include the students who undoubtedly failed (≤41 out of 100), and S1 would include those who passed (≥50 out of 100) as well as those who were close to the pass mark (>41 but <50 out of 100), and deserved, subjectively, “an opportunity”. The students showed the following distribution: Mechanical Engineering (S1/S2: 16/14), Electrical Engineering (S1/S2: 11/4), Industrial Electronic Engineering (S1/S2: 20/7) and Industrial Chemical Engineering (S1/S2: 9/4). 3) Formal assessments The third instrument used for comparing purposes consists of two formal assessments (ESP Survey One and ESP Survey Two). These students can sit for one and/or for the other. Those from the sample sat for both examinations. The structure of the formal assessments focuses mainly on tasks an engineer might fulfill in a local firm, that is, reading comprehension on topics connected to his/her branch of engineering (e.g. how to remove the cylinder head in a diesel engine and what a close scrutiny of the combustion chamber could reveal –high oil consumption, over-fueling, overheating, etc.), listening comprehension (e.g. listen to the following descriptions:… Now, match them to these instruments or tools: a voltmeter, an oscilloscope, a heat sink, wire-clippers, a megger…), or tasks the student might need to accomplish (e.g. study some advertisements and select a suitable job for which an applicant could apply, and write a letter of application). In order to test HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 159 08 JOSEBA.qxp 6/4/07 09:07 Página 159 writing, the test takers are asked to develop some kind of text (e.g. list the main elements you need for assembling a universal motor and explain the function of each element). Language functions are also tested (e.g. expressing reason: Fill in the gaps inserting one of the following prepositions –for, out of, from, with– “He did it … gratitude”, “I know it … experience”, “I’m just asking … interest”, “The plants died … want of water”,…). Speaking is also evaluated –monologue or dialogue– (e.g. describe how the pH of a certain fluid is measured). Most tasks are familiar to the students through their content lecturers but the linguistic tools to accomplish them in the target language are provided by the ESP lecturer. Overall marks range from 0 to 100 (both limits are very uncommon), and a normal distribution is regularly observed. The validity period of scores –Grammar Survey One and Two– is considered to be long enough (>2 semesters) to affect performance positively in ESP Survey One and Two. The results of our students before taking the ESP courses (readiness tests), will be compared with those obtained after taking them (formal assessments). The information obtained from the questionnaire will be used to try to explain the existing correlations between both groups of tests. Results Descriptive analyses Student Subgroup One (S1) and Student Subgroup Two (S2) are the names given to the subgroups into which the entire group was divided, i.e. those students considered potentially valid to take the ESP courses (S1), and those whose command of general English was not, in theory, high enough for completing the courses successfully (S2). In order to carry out the statistical analyses corresponding to the data obtained, the SPSS (Statistical Package for the Social Sciences) program was used. JOSEBA M. GONZÁLEZ ARDEO 160 IBÉRICA 13 [2007]: 147-170 JOSEBA M. GONZÁLEZ ARDEO The validity period of scores !Grammar Survey One and Two! is considered to be long enough (>2 semesters) to affect performance positively in ESP Survey One and Two. The results of our students before taking the ESP courses (readiness tests), will be compared with those obtained after taking them (formal assessments). The information obtained from the questionnaire will be used to try to explain the existing correlations between both groups of tests. Results Descriptive analyses Student Subgroup One (S1) and Student Subgroup Two (S2) are the names given to the subgroups into which the entire group was divided, i.e. those students considered potentially valid to take the ESP courses (S1), and those whose command of general English was not, in theory, high enough for completing the courses successfully (S2). In order to carry out the statistical analyses corresponding to the data obtained, the SPSS (Statistical Package for the Social Sciences) program was used. S GSO + GST SSO SST S1 (n=56) S2 (n=29) 82.28 47.23 62.14 46.75 60.64 44.34 Table 1. Mean marks per group of students. At first glance (Table 1), there is a clear correlation between the marks attained, both in ESP Survey One and Two, by students from S1 and S2. As a whole, students from S1 pass both examinations, ESP Survey One and Two, while students from S2 fail both. Table 2 shows the items included in the questionnaire, the options chosen within each item (see Appendix I), the marks obtained in Grammar Survey One plus Two, and the marks obtained in ESP Survey One and Two. It is difficult to compare learners of different ages if the contexts and environments in which learning may take place are considered, but the overall experience accumulated n = number of individuals GSO = Grammar Survey One (max. 116) GST = Grammar Survey Two (max. 30) SSO = ESP Survey One (max. 100) SST = ESP Survey Two (max. 100) 08 JOSEBA.qxp 6/4/07 09:07 Página 160 SAt first glance (Table 1), there is a clear correlation between the marks attained, both in ESP Survey One and Two, by students from S1 and S2. As a whole, students from S1 pass both examinations, ESP Survey One and Two, while students from S2 fail both. Table 2 shows the items included in the questionnaire, the options chosen within each item (see Appendix I), the marks obtained in Grammar Survey One plus Two, and the marks obtained in ESP Survey One and Two. It is difficult to compare learners of different ages if the contexts and environments in which learning may take place are considered, but the overall experience accumulated by a student at the age of 19 and that s/he may have accumulated at the age of 23/24 may differ significantly. This is the reason why performances with the age variable were compared. It can be observed that the mean marks of the different age groups differ importantly when mean values of Grammar Survey One plus Two are compared (see Table 2). However, the results do not differ that much when ESP Surveys are compared. The best overall results coincide with ages 20 and 21, and the worst with the oldest individuals. Table 2 also shows the following: 1) differences between men and women (at first sight, female students outperformed male ones, but the margin is rather narrow); 2) bilinguals (18+25) outperformed monolinguals (42) by a narrow margin; 3) only 10 students out of 85 considered themselves introverted (a priori surprisingly, these students outperformed the extroverts); 4) most students come from the city or from towns, while just a few come from rural settings (the differences between them are negligible); 5) the differences between both groups in terms of performance are not clearly disparate when “educational level” is considered; 6) those students who had voluntarily taken English courses as well as compulsory ones outperformed rather clearly those who had only taken compulsory courses; 7) no unfavorable attitudes towards ESP are expressed by the students from the sample and, at first sight, the-more-favorable-attitude-you-have-the-better seems to be true; 8) most students (73 out of 85) describe their level of motivation as simply “motivated” (the more motivated the students are, the better they apparently perform, but this is neither importantly nor always true in certain cases); 9) only those students who considered their former teachers “very efficient” passed (with credit) all the tests and assessments (some of the results obtained by those who chose the option “very deficient” are rather surprising); 10) those who describe their language aptitude as “very positive” obtained outstanding results in comparison with the others; 11) present HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 161 08 JOSEBA.qxp 6/4/07 09:07 Página 161 performance seems to be undoubtedly conditioned by past performance; 12) only those students who considered their communication aptitude “very good” clearly outperformed the others; 13) the differences between the groups are not very important unless those differences that take place between the first and the last group are observed; and 14) only two students had attended a trilingual program (both performed extremely well in all Surveys). When the results obtained by the students are grouped according to their branch of engineering (Table 3), no striking differences amongst the branches are observed. JOSEBA M. GONZÁLEZ ARDEO 162 IBÉRICA 13 [2007]: 147-170 HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: …-… 13 Q Options GSO+GST SSO SST Age (Q1) 1 (n=15) 2 (n=28) 3 (n=20) 4 (n=17) 5 (n=5) 68.85 74.59 78.65 58.10 59.10 55.53 59.39 61.55 50.82 49.00 53.13 58.14 55.00 52.52 52.80 Gender (Q2) 1 (n=48) 2 (n=37) 68.44 72.76 56.56 57.32 53.58 57.02 Mother tongue (Q3) 1 (n=18) 2 (n=42) 3 (n=25) 71.81 68.81 71.79 58.77 54.52 59.52 57.77 54.09 54.80 Personality trait (Q4) 1 (n=10) 2 (n=75) 74.50 69.77 62.40 56.16 57.00 54.82 Social setting (Q5) 1 (n=16) 2 (n=69) 69.40 70.53 59.37 56.31 54.87 55.13 Educational level (Q6) 1 (n=10) 2 (n=75) 72.65 70.01 61.20 56.32 54.30 55.18 Language background (Q7) 1 (n=75) 2 (n=10) 68.77 82.00 55.94 64.00 53.70 65.40 Attitude towards ESP-EST (Q8) 1 (n=10) 2 (n=62) 3 (n=13) 85.00 68.65 67.00 62.30 56.20 56.00 65.50 53.14 56.30 Motivation (Q9) 1 (n=8) 2 (n=73) 3 (n=4) 103.13 66.84 68.38 71.13 55.42 55.25 66.50 53.99 52.25 Former teachers’ efficiency (Q10) 1 (n=6) 2 (n=71) 3 (n=5) 4 (n=3) 80.16 69.00 73.05 77.50 66.00 55.49 62.00 63.33 65.66 54.21 59.40 47.33 Language learning aptitude (Q11) 1 (n=8) 2 (n=73) 3 (n=4) 111.31 65.97 67.68 80.62 54.36 55.50 73.62 53.05 55.00 Previous academic performance (Q12) 1 (n=7) 2 (n=71) 3 (n=7) 127.00 67.76 39.64 84.85 55.97 38.28 79.71 54.36 37.71 Communicative attitude (Q13) 1 (n=7) 2 (n=64) 3 (n=12) 4 (n=2) 109.57 71.02 47.58 47.12 76.28 57.76 41.66 52.50 75.00 55.98 40.41 44.50 Phonetic coding ability (Q14) 1 (n=5) 2 (n=58) 3 (n=19) 4 (n=3) 77.90 74.75 58.57 46.58 59.60 59.13 50.78 47.66 62.20 56.13 52.26 40.66 English means of instruction (Q15) 0 (n=83) 3 (n=2) 69.12 120.37 56.01 93.50 54.09 96.00 Table 2. Mean values of performances depending on questions Q1 to Q15 and their options Table 2 also shows the following: 1) differences between men and women (at first sight, female students outperformed male ones, but the margin is rather narrow); 2) bilinguals (18+25) outperformed monolinguals (42) by a narrow margin; 3) only 10 students out of 85 considered themselves introverted (a priori surprisingly, these students outperformed the extroverts); 4) most students come 08 JOSEBA.qxp 6/4/07 09:07 Página 162 Normality tests were carried out for all quantitative variables (see Table 4). It can be observed that the five variables fulfill normality. Model diagnosis The purpose of this section is to check what variables the marks obtained in ESP Survey One and Two depend upon. The independent variables considered were Q1, Q2, Q3, Q4, Q5, Q6, Q7, Q8, Q9, Q10, Q11, Q12, Q13 and Q14, as well as the re-codified variable referred to the group of students S (S1 if marks of GSO+GST≥60; S2 if marks of GSO+GST<60). Q15 was not considered due to the almost constant distribution of data. In order to simplify calculations, ANOVA (ANalysis Of VAriance) analyses with a single independent factor were carried out first. The candidate variables to be part of the final model are given by the result of these analyses. The analyses were independently undertaken for both variables, ESP Survey One and Two. Model analysis for dependent variable “ESP Survey One” Inter-individual ANOVA analyses were carried out for each and every independent variable, one by one (except for Q15 as previously mentioned). Taking a .05 significance level as a reference, the values taken by variables S, Q9, Q11, Q12, and Q13 create significant differences in the marks of ESP Survey One. A new ANOVA analysis with these five independent variables as a whole, but without interactions, was carried out: HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 163 IBÉRICA 13 [2007]: …-…14 those students who considered their communication aptitude “very good” clearly outperformed the others; 13) the differences between the groups are not very important unless those differences that take place between the first and the last group are observed; and 14) only two students had attended a trilingual program (both performed extremely well in all Surveys). When the results obtained by the students are grouped according to their branch of engineering (Table 3), no striking differences amongst the branches are observed. BRANCH GSO+GST SSO SST MechEng (n=30) 66.63 57.47 55.20 ElecEng (n=15) 71.83 58.40 57.87 IndElecEng (n=26) 74.03 56.00 54.00 IndChemEng (n=14) 69.75 55.71 53.86 Total (n=85) 70.33 56.89 55.08 Table 3. Mean marks per branch of engineering. Normality tests were carried out for all quantitative variables (see Table 4). GSO GST GSO+GST SSO SST n Normal parameter a, Mean Typical deviation Kolmogorov-Smirnov’s Z Asymptotic sig. (bilateral) 85 55.9529 19.96863 1.072 .201 85 14.3735 4.70809 .972 .301 85 70.3265 24.23702 1.054 .216 85 56.8941 15.74879 1.275 .077 85 55.0824 17.01555 .707 .699 a. The contrast distribution is the Normal one. Table 4. Kolmogorov-Smirnov test for a sample. It can be observed that the five variables fulfill normality. IBÉRICA 13 [2007]: …-…14 obtained outstanding results in comparison with the others; 11) present performance seems to be undoubtedly conditioned by past performance; 12) only those students who considered their communication aptitude “very good” clearly outperformed the others; 13) the differences between the groups are not very important unless those differences that take place between the first and the last group are observed; and 14) only two students had attended a trilingual program (both performed extremely well in all Surveys). When the results obtained by the students are grouped according to their branch of engineering (Table 3), no striking differences amongst the branches are observed. BRANCH GSO+GST SSO SST MechEng (n=30) 66.63 57.47 55.20 ElecEng (n=15) 71.83 58.40 57.87 IndElecEng (n=26) 74.03 56.00 54.00 IndChemEng (n=14) 69.75 55.71 53.86 Total (n=85) 70.33 56.89 55.08 Table 3. Mean marks per branch of engineering. Normality tests were carried out for all quantitative variables (see Table 4). GSO GST GSO+GST SSO SST n Normal parameter a, Mean Typical deviation Kolmogorov-Smirnov’s Z Asymptotic sig. (bilateral) 85 55.9529 19.96863 1.072 .201 85 14.3735 4.70809 .972 .301 85 70.3265 24.23702 1.054 .216 85 56.8941 15.74879 1.275 .077 85 55.0824 17.01555 .707 .699 a. The contrast distribution is the Normal one. Table 4. Kolmogorov-Smirnov test for a sample. It can be observed that the five variables fulfill normality. 08 JOSEBA.qxp 6/4/07 09:07 Página 163 It is observed (see Table 5) that although the model is correct, certain variables do not sufficiently discriminate the values of ESP Survey One enough so as to be included in the final model. The variables with a higher significance value (Q9, Q11, and Q13) were eliminated, one by one. As a result of this process, the significance value is now below .05 for all the variables (see Table 6). Significance Therefore, it can be stated that the marks obtained in assessment ESP Survey One will depend upon variables S and Q12, namely, the marks obtained in Grammar Survey One and Two and the PAP of the students. As a result of studying this dependence and after estimating the different variables, it can be observed that the best marks in ESP Survey One belong to those students that fulfill the following conditions: 1) Grammar Survey One + Grammar Survey Two ≥ 60; 2) Very good PAP. Conversely, the students who attained the worst marks were those who got less than 60 marks (GSO+GST) and whose PAP was bad. This evidence is corroborated by results from Table 7, i.e. the mean values the students would attain for each category of variables S and Q12. JOSEBA M. GONZÁLEZ ARDEO 164 IBÉRICA 13 [2007]: 147-170 IBÉRICA 13 [2007]: …-… 15 Model analysis for dependent variable “ESP Survey One” Inter-individual ANOVA analyses were carried out for each and every independent variable, one by one (except for Q15 as previously mentioned). Taking a .05 significance level as a reference, the values taken by variables S, Q9, Q11, Q12, and Q13 create significant differences in the marks of ESP Survey One. A new ANOVA analysis with these five independent variables as a whole, but without interactions, was carried out: Significance Corrected model Intersection S Q9 Q11 Q12 Q13 .000 .000 .009 .902 .427 .010 .123 Table 5. Significance values of independent variables S, Q9, Q11, Q12 and Q13. It is observed (see Table 5) that although the model is correct, certain variables do not sufficiently discriminate the values of ESP Survey One enough so as to be included in the final model. The variables with a higher significance value (Q9, Q11, and Q13) were eliminated, one by one. As a result of this process, the significance value is now below .05 for all the variables (see Table 6). Significance Corrected model Intersection S Q12 .000 .000 .003 .000 Table 6. Significance values of independent variables S and Q12. Therefore, it can be stated that the marks obtained in assessment ESP Survey IBÉRICA 13 [2007]: …-… 15 In order to simplify calculations, ANOVA (ANalysis Of VAriance) analyses with a single independent factor were carried out first. The candidate variables to be part of the final model are given by the result of these analyses. The analyses were independently undertaken for both variables, ESP Survey One and Two. Model analysis for dependent variable “ESP Survey One” Inter-individual ANOVA analyses were carried out for each and every independent variable, one by one (except for Q15 as previously mentioned). Taking a .05 significance level as a reference, the values taken by variables S, Q9, Q11, Q12, and Q13 create significant differences in the marks of ESP Survey One. A new ANOVA analysis with these five independent variables as a whole, but without interactions, was carried out: Significance Corrected model Intersection S Q9 Q11 Q12 Q13 .000 .000 .009 .902 .427 .010 .123 Table 5. Significance values of independent variables S, Q9, Q11, Q12 and Q13. It is observed (see Table 5) that although the model is correct, certain variables do not sufficiently discriminate the values of ESP Survey One enough so as to be included in the final model. The variables with a higher significance value (Q9, Q11, and Q13) were eliminated, one by one. As a result of this process, the significance value is now below .05 for all the variables (see Table 6). Significance Corrected model Intersection S Q12 .000 .000 .003 .000 Table 6. Significance values of independent variables S and Q12. Therefore, it can be stated that the marks obtained in assessment ESP Survey JOSEBA M. GONZÁLEZ ARDEO One will depend upon variables S and Q12, namely, the marks obtained in Grammar Survey One and Two and the PAP of the students. As a result of studying this dependence and after estimating the different variables, it can be observed that the best marks in ESP Survey One belong to those students that fulfill the following conditions: 1) Grammar Survey One + Grammar Survey Two ! 60; 2) Very good PAP. Conversely, the students who attained the worst marks were those who got less than 60 marks (GSO+GST) and whose PAP was bad. This evidence is corroborated by results from Table 7, i.e. the mean values the students would attain for each category of variables S and Q12. INTERSECTION S Q12 ESP Survey one 38.286 [S=1] [S=2] 9.443 0.000 [Q12]=1 [Q12]=2 [Q12]=3 [Q12]=1 [Q12]=2 [Q12]=3 37.12 11.16 00.00 37.12 11.16 00.00 84.85 58.89 47.72 75.41 49.45 38.28 Table 7. Mean values per category of variables S and Q12. Model analysis for dependent variable “ESP Survey Two” One again, as many inter-individual ANOVA analyses as independent variables were carried out (except for Q15). For a .05 significance value, variables S, Q7, Q9, Q11, Q12, and Q13 create significant differences in the marks of ESP Survey Two. The analysis with these five independent variables as a whole, without interactions, shows the following: 08 JOSEBA.qxp 6/4/07 09:07 Página 164 Model analysis for dependent variable “ESP Survey Two” Once again, as many inter-individual ANOVA analyses as independent variables were carried out (except for Q15). For a .05 significance value, variables S, Q7, Q9, Q11, Q12, and Q13 create significant differences in the marks of ESP Survey Two. The analysis with these five independent variables as a whole, without interactions, shows the following: The model is correct (Table 8) but some variables do not discriminate the values of ESP Survey Two enough so as to be included in the final model. In order to achieve this, the variables with a higher significance value were eliminated one by one, starting with Q11, since this one showed the highest significance value. Later on, Q13, Q9, and Q7 were eliminated. Now, the significance values are below .05 (see Table 9). The conclusion reached by following this process is similar to the one obtained for the variable ESP Survey One, that is, the marks obtained in ESP Survey Two will depend upon variables S and Q12. Again, the best marks are attained by those students who fulfill the same two previous conditions: 1) Grammar Survey One + Grammar Survey Two ≥ 60; 2) Very good PAP. The worst marks are for students who: 1) GSO+GST<60; 2) PAP = bad. When this dependence is studied and after estimating the different variables, the best marks in ESP Survey Two, as in ESP Survey One, belong to those students that fulfill: 1) Grammar Survey One + Grammar Survey Two ≥ 60: 2) Very good PAP. Once again, this evidence is corroborated by results from Table 10, i.e. the mean values the students would attain for each category of variables S and Q12. HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 165 IBÉRICA 13 [2007]: …-…16 Q12. INTERSECTION S Q12 ESP Survey one 38.286 [S=1] [S=2] 9.443 0.000 [Q12]=1 [Q12]=2 [Q12]=3 [Q12]=1 [Q12]=2 [Q12]=3 37.12 11.16 00.00 37.12 11.16 00.00 84.85 58.89 47.72 75.41 49.45 38.28 Table 7. Mean values per category of variables S and Q12. Model analysis for dependent variable “ESP Survey Two” One again, as many inter-individual ANOVA analyses as independent variables were carried out (except for Q15). For a .05 significance value, variables S, Q7, Q9, Q11, Q12, and Q13 create significant differences in the marks of ESP Survey Two. The analysis with these five independent variables as a whole, without interactions, shows the following: Significance Corrected model Intersection S Q7 Q9 Q11 Q12 Q13 .000 .000 .020 .252 .411 .894 .081 .533 Table 8. Significance values of independent variables S, Q7, Q9, Q11, Q12 and Q13. The model is correct (Table 8) but some variables do not discriminate the values of ESP Survey Two enough so as to be included in the final model. In order to achieve this, the variables with a higher significance value were eliminated one by one, starting with Q11, since this one showed the highest significance value. Later on, Q13, Q9, and Q7 were eliminated. Now, the significance values are below .05 (see Table 9). HOW LINGUISTICALLY READY Significance Corrected model Intersection S Q12 .000 .000 .002 .000 Table 9. Significance values of independent variables S and Q12. The conclusion reached by following this process is similar to the one obtained for the variable ESP Survey One, that is, the marks obtained in ESP Survey Two will depend upon variables S and Q12. Again, the best marks are attained by those students who fulfill the same two previous conditions: 1) Grammar Survey One + Grammar Survey Two ! 60; 2) Very good PAP. The worst marks are for students who: 1) GSO+GST<60; 2) PAP = bad. When this dependence is studied and after estimating the different variables, the best marks in ESP Survey Two, as in ESP Survey One, belong to those students that fulfill: 1) Grammar Survey One + Grammar Survey Two ! 60: 2) Very good PAP. Once again, this evidence is corroborated by results from Table 10, i.e. the mean values the students would attain for each category of variables S and Q12. INTERSECTION S Q12 ESP Survey one 37.714 [S=1] [S=2] 11.46 00.00 [Q12]=1 [Q12]=2 [Q12]=3 [Q12]=1 [Q12]=2 [Q12]=3 30.53 08.74 00.00 30.53 08.74 00.00 79.71 57.91 49.17 68.25 46.45 37.71 Table 10. Mean values per category of variables S and Q12. Factorial analysis of main components An overall vision of the problem can be achieved by means of the Main 08 JOSEBA.qxp 6/4/07 09:07 Página 165 Factorial analysis of main components An overall vision of the problem can be achieved by means of the Main Components method, factorial analysis, for the quantitative variables ESP Survey One, ESP Survey Two and Grammar Survey One + Grammar Survey Two. This gives a two-dimensional view of how these three variables (SSO, SST, GSO+GST) could be represented, and to study these variables with respect to the most representative variables (S, Q1 … Q14). Bartlett’s test is carried out first in order to apply the Main Components method. It can be observed (see Table 11) that the three variables (GSO+GST, SSO and SST) are explained by the first component. The fact that a single variable (component) is enough for explaining the three variables indicates a close correspondence or relationship amongst them. It was assumed that a close relationship between the variables ESP Survey One, and ESP Survey Two, and the variable Q12 would exist. If the cases are labeled by means of Q12 (see Graph 1), it can be clearly observed that high values in the first component (high values in SSO, SST, and GSO+GST) correspond to those students that chose option 1 in Q12 JOSEBA M. GONZÁLEZ ARDEO 166 IBÉRICA 13 [2007]: 147-170 IBÉRICA 13 [2007]: …-… 17 One + Grammar Survey Two ! 60; 2) Very good PAP. The worst marks are for students who: 1) GSO+GST<60; 2) PAP = bad. When this dependence is studied and after estimating the different variables, the best marks in ESP Survey Two, as in ESP Survey One, belong to those students that fulfill: 1) Grammar Survey One + Grammar Survey Two ! 60: 2) Very good PAP. Once again, this evidence is corroborated by results from Table 10, i.e. the mean values the students would attain for each category of variables S and Q12. INTERSECTION S Q12 ESP Survey one 37.714 [S=1] [S=2] 11.46 00.00 [Q12]=1 [Q12]=2 [Q12]=3 [Q12]=1 [Q12]=2 [Q12]=3 30.53 08.74 00.00 30.53 08.74 00.00 79.71 57.91 49.17 68.25 46.45 37.71 Table 10. Mean values per category of variables S and Q12. Factorial analysis of main components An overall vision of the problem can be achieved by means of the Main Components method, factorial analysis, for the quantitative variables ESP Survey One, ESP Survey Two and Grammar Survey One + Grammar Survey Two. This gives a two-dimensional view of how these three variables (SSO, SST, GSO+GST) could be represented, and to study these variables with respect to the most representative variables (S, Q1 … Q14). Bartlett’s test is carried out first in order to apply the Main Components method. It can be observed (see Table 11) that the three variables (GSO+GST, SSO and SST) are explained by the first component. The fact that a single variable (component) is enough for explaining the three variables indicates a close correspondence or relationship amongst them. JOSEBA M. GONZÁLEZ ARDEO Kaiser-Meyer-Olkin’s sample fitting measurement .702 Communalities Initial Extraction Bartlett’s sphericity test: Approx. Chi-squared df Sig. 160.517 3 .000 GSO+GST SSO SST 1.000 1.000 1.000 .979 .897 .962 Extraction method: Main Components analysis Total variance explained Components matrix a Components Total % of variance % accumulated 1 2 1 2 3 2.485 .353 .162 82.843 11.774 5.383 82.843 94.617 100.000 GSO+GST SSO SST .885 .946 .898 .443 -3.9E-02 -.394 Extraction method: Main Components analysis Extraction method: Main Components analysis a. Two components extracted Table 11. KMO and Bartlett’s test. It was assumed that a close relationship between the variables ESP Survey One, and ESP Survey Two, and the variable Q12 would exist. If the cases are labeled by means of Q12 (see Graph 1), it can be clearly observed that high values in the first component (high values in SSO, SST, and GSO+GST) correspond to those students that chose option 1 in Q12 (continuous arrow in Graph 1). Conversely, negative values in the first component (low values in SSO, SST, and GSO+GST) correspond to students that chose option 3 in Q12 (discontinuous arrow in Graph 1). 3 2 1 2 3 2 2 2 2 3 2 22 1 1 2 2 2 2 2 2 2 2 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 1 JOSEBA M. GONZÁLEZ ARDEO Kaiser-Meyer-Olkin’s sample fitting measurement .702 Communalities Initial Extraction Bartlett’s sphericity test: Approx. Chi-squared df Sig. 160.517 3 .000 GSO+GST SSO SST 1.000 1.000 1.000 .979 .897 .962 Extraction method: Main Components analysis Total variance explained Components matrix a Components Total % of variance % accumulated 1 2 1 2 3 2.485 .353 .162 82.843 11.774 5.383 82.843 94.617 100.000 GSO+GST SSO SST .885 .946 .898 .443 -3.9E-02 -.394 Extraction method: Main Components analysis Extraction method: Main Components analysis a. Two components extracted Table 11. KMO and Bartlett’s test. It was assumed that a close relationship between the variables ESP Survey One, and ESP Survey Two, and the variable Q12 would exist. If the cases are labeled by means of Q12 (see Graph 1), it can be clearly observed that high values in the first component (high values in SSO, SST, and GSO+GST) correspond to those students that chose option 1 in Q12 (continuous arrow in Graph 1). Conversely, negative values in the first component (low values in SSO, SST, and GSO+GST) correspond to students that chose option 3 in Q12 (discontinuous arrow in Graph 1). 3 2 1 2 2 2 2 2 2 1 1 2 2 2 2 2 2 2 1 1 1 1 2 2 2 2 2 22 2 2 2 2 1 08 JOSEBA.qxp 6/4/07 09:07 Página 166 (continuous arrow in Graph 1). Conversely, negative values in the first component (low values in SSO, SST, and GSO+GST) correspond to students that chose option 3 in Q12 (discontinuous arrow in Graph 1). Conclusions ESP Survey One and Two assess, in theory, both general English proficiency and the ability to use “engineering” English. A restricted topic domain could result in an invalid “proficiency” rating. For this reason, these formal assessments try to be as integrative as possible, minimize subjectivity when scoring, and include as realistic tasks as possible. In these days of “communicative testing” these three conditions seem to be the recipe for success when assessing knowledge in a foreign language. Grammar Survey One and Two only have a few similarities with ESP Survey One and Two, but the information provided by the former was necessary to give a proper answer to the title of this paper: How linguistically ready are my engineering students to take my ESP courses? Then, considering the tests carried out and the results obtained, the following conclusions can be stated: 1) the success of our students can be rather easily predicted once Grammar Survey One and Two are taken and information about their Previous Academic Performance is gathered; 2) the marks obtained in the formal assessments, ESP Survey One and Two, depend upon the marks obtained in Grammar Survey One and Two, as well as upon the Previous Academic Performance of the students. HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 167 IBÉRICA 13 [2007]: …-…18 3 .162 5.383 100.000 SST .898 -.394 Extraction method: Main Components analysis Extraction method: Main Components analysis a. Two components extracted Table 11. KMO and Bartlett’s test. It was assumed that a close relationship between the variables ESP Survey One, and ESP Survey Two, and the variable Q12 would exist. If the cases are labeled by means of Q12 (see Graph 1), it can be clearly observed that high values in the first component (high values in SSO, SST, and GSO+GST) correspond to those students that chose option 1 in Q12 (continuous arrow in Graph 1). Conversely, negative values in the first component (low values in SSO, SST, and GSO+GST) correspond to students that chose option 3 in Q12 (discontinuous arrow in Graph 1). 3210-1-2 3 2 1 0 -1 -2 2 2 2 3 2 2 3 2 2 2 2 2 3 2 2 2 2 2 2 2 3 3 2 2 2 2 2 2 2 2 2 2 2 2 3 1 1 2 2 2 2 2 2 2 2 2 2 23 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 22 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 Graph 1. Cases labeled by means of Q12. 08 JOSEBA.qxp 6/4/07 09:07 Página 167 The feeling that the battle is lost before it begins, because of failings in previous language study and the poor level of knowledge with which some students come to the university, is founded on facts. In other words, the results obtained confirm main hypotheses 1.1 (Students will exhibit appreciable differences when tested on specific language learning, depending upon their previous academic performance within the field of English for General Purposes.) and 1.2 (Grammar Surveys carried out prior to the ESP courses will tell us in advance which students would pass or fail the ESP tests the first time around.), as well as secondary hypothesis 2.1 (Variables such as age, gender, mother tongue, personality traits, social setting and educational level, will not affect performance in the ESP courses administered.). This is not the case with hypothesis 2.2 (Variables such as language background, attitude towards ESP, motivation, former teachers’ efficiency, language learning aptitude, previous academic performance (PAP), communicative aptitude, and phonetic coding ability will affect performance in the ESP courses and their corresponding assessments.), since most variables, all but one (PAP)., evaluated through this hypothesis do not significantly affect performance in ESP Survey One and Two. A straight-forward corollary could be as simple as this: students should take Grammar Survey One and Two (or any equivalent test) prior to enrolment and then take or not take the courses depending on the marks obtained so that success can be guaranteed to a certain extent. Moreover, if their Previous Academic Performance was bad, they are potential candidates for failure. It seems then that, no matter how efficient the lecturer is, the students are linguistically predestined in our ESP courses, and the teacher’s share of responsibility in their failures appears to be negligible. Acknowledgements I would like to thank the following colleagues for their valuable comments on earlier drafts of this paper: Patricia Edwards Rokowsky and José Castro Calvín. Any absurdities that may remain are, of course, my own. (Revised paper received November 2006) JOSEBA M. GONZÁLEZ ARDEO 168 IBÉRICA 13 [2007]: 147-170 08 JOSEBA.qxp 6/4/07 09:07 Página 168 REFERENCES Dr. Joseba M. González Ardeo (Bachelor of Engineering; Master in Foreign Trade; Ph.D. in Organizational Communication; Master in TEFL) is Associate Professor and teaches ESP at the University of the Basque Country. His current research priorities fall within the scope of multilingualism and English for Specific/Academic Purposes. HOW LINGUISTICALLY READY IBÉRICA 13 [2007]: 147-170 169 Albert, M. L. & L.K. Obler (1978). The Bilingual Brain. New York: Academic. Bachman, L. (1990). Fundamen- tal Considerations in Language Testing. Oxford: Oxford Univer- sity Press. Brown, J. D. (1978). English as a Second Language Placement Examination. Los Angeles, CA: TESL Section English Depart- ment, University of California. Brown, J. D. (1983). “A closer look at cloze: validity and reli- ability” in J. W. Oller, Jr. (ed.), Issues in Language Testing Re- search, 237-250. Rowley, MA: Newbury House. Brown, S., C. Rust & G. Gibbs (1994). Strategies for Diversify- ing Assessment in Higher Edu- cation. Oxford: Oxford Centre for Staff Development. Burstall, C. (1975). “Factors af- fecting foreign language learn- ing: A consideration of some re- cent research findings”. Lan- guage Teaching 8: 1-21. Carroll, J. & S. Sapon (1959). Modern Language Aptitude Test-Form A. New York: The Psychological Corporation. Cenoz, J. & F. Genesse (1998). “Psycholinguistic perspectives on multilingualism and multilin- gual education” in J. Cenoz & F. Genesse (eds.), Beyond Bilin- gualism: Multilingualism and Multilingual Education, 16-32. Clevedon: Multilingual Matters. Clark, J. (1987). Curriculum Re- newal in School Foreign Lan- guage Learning. Oxford: Oxford University Press. Doughty, C. & J. Williams (1998). “Pedagogical choice in focus on form” in C. Doughty & J. Williams (eds.), Focus on Form in Classroom Second Lan- guage Acquisition, 197-261. Cambridge: Cambridge Univer- sity Press. Ehrman, M. & R. Oxford (1990). “Adult language learning styles and strategies in an intensive training setting”. Modern Lan- guage Journal, 74: 311-327. Ellis, R. (1985). Understanding Second Language Acquisition. Oxford: Oxford University Press. Gardner, R. (1985). Social Psy- chology and Second Language Learning: The Role of Attitudes and Motivation. Rowley, MA: Newbury House. Genesse, F. (1976). “The role of intelligence in second language learning”. Language Learning 26: 267-280. Giles, H. & W.P. Robinson (eds.) (1990). Handbook of Language and Social Psychology. Chich- ester: Wiley. González, J. M. (2003). “Attitude towards English and ESP acqui- sition as an L2 or L3 at universi- ty”. Ibérica 6: 109-133. González, J. M. (2004). “ESP courses and linguistic achieve- ment of engineering students in the Basque Country”. VIAL 1: 75-110. Kunnan, A. J. (ed.) (2000). Fair- ness and Validation in Language Assessment. Cambridge: UCLES. Lightbown, P. M. & N. Spada (1993). How Languages are Learned. Oxford: Oxford Univer- sity Press. Liski, E. & S. Putanen (1983). “A study of the statistical founda- tions of group conversation tests of spoken English”. Language Learning 33: 225-246. Oxford, R. & M. Nyikos (1989). “Variables affecting choice of language learning strategies by university students”. Modern Language Journal 73: 291-300. Purpura, J. E. (2004). Assessing Grammar. Cambridge: Cam- bridge University Press. Ringbom, H. (1985). Foreign Language Learning and Bilin- gualism. Finland: Abo. Skehan, P. (1989). Individual Differences in Second Lan- guage Acquisition. London: Arnold. Swan, M. & C. Walter (1997). How English Works. A Grammar Practice Book. Oxford: Oxford University Press. Wenden, A. (1986). “What do second language learners know about their language learning? A second look at retrospective ac- counts”. Applied Linguistics 7: 186-205. 08 JOSEBA.qxp 6/4/07 09:07 Página 169 Appendix I (Variables) Questionnaire and options. JOSEBA M. GONZÁLEZ ARDEO 170 IBÉRICA 13 [2007]: 147-170 JOSEBA M. GONZÁLEZ ARDEO IBÉRICA 13 [2007]: …-…22 BRAN Branch of Engineering [(MechEng)/(ElecEng)/(IndElecEng)/(IndChemEng)] Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Age [19(1)/20(2)/21(3)/22(4)/23-24(5)] Gender [male(1)/female(2)] Mother tongue [Basque(1)/Spanish(2)/Basque/Spanish(3)/other(4)] Personality trait [introverted(1)/extrovert(2)] Social setting [rural(1)/city-town(2)] Educational level [Vocational training(1)/Higher certificate(2)/Over 25(3)] Language background [Compulsory English courses at school(1)/Compulsory English courses at school + voluntary English courses(2)] Attitude towards ESP [Very favourable(1)/Favourable(2)/Neutral(3)/Unfavourable(4)/Very unfavourable (5)] Motivation [Very motivated(1)/Motivated(2)/Low motivation(3)/No motivation(4)] Former teachers’ efficiency [Very efficient(1)/Efficient(2)/Deficient(3)/Very deficient(4)] Language learning aptitude [Very positive(1)/Positive(2)/Negative(3)/Very negative(4)] Previous academic performance [Very good(1)/Fair(2)/Bad(3)/Very bad(4)] Communicative aptitude [Very good(1)/Fair(2)/Bad(3)/Very bad(4)] Phonetic coding ability [Very high(1)/High(2)/Average(3)/Low(4)/Very low(5)] Has English ever been a means of instruction in subjects other than ‘English’? (EIS) [No(0)/Yes All my subjects were taught in English(1) My school programme was bilingual –English/Spanish or Basque-(2) My school programme was trilingual –English/Spanish/Basque-(3) Others(4)] 08 JOSEBA.qxp 6/4/07 09:07 Página 170