Akkoyunlu, A. N., & Kilimci, A. (2017). Application of corpus to translation teaching: Practice and perceptions. International Online Journal of Education and Teaching (IOJET), 4(4). 369-396. http://iojet.org/index.php/IOJET/article/view/272/178 Received: 11.08.2017 Received in revised version: 14.08.2017 Accepted: 21.08.2017 APPLICATION OF CORPUS TO TRANSLATION TEACHING: PRACTICE AND PERCEPTIONS Aslı Nur Akkoyunlu Ministry of National Education, Turkey asliakkoyunlu@gmail.com Abdurrahman Kilimci Çukurova University, Adana, Turkey akilimci@cu.edu.tr Aslı Nur Akkoyunlu holds an MA degree in English Language Teaching and is currently an English teacher at the Ministry of National Education. Her research interests are applied linguistics, corpus linguistics and second language acquisition. Dr. Abdurrahman Kilimci is affiliated with Çukurova University, Adana, and works as an assistant professor at the Faculty of Education, English Language Teaching Department. His research interests are applied linguistics, corpus linguistics, cognitive linguistics and second language acquisition. Copyright by Informascope. Material published and so copyrighted may not be published elsewhere without the written permission of IOJET. http://iojet.org/index.php/IOJET/article/view/243/174 http://orcid.org/0000-0002-2955-968X http://orcid.org/0000-0002-3867-2369 International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 369 APPLICATION OF CORPUS TO TRANSLATION TEACHING: PRACTICE AND PERCEPTIONS * Aslı Nur Akkoyunlu asliakkoyunlu@gmail.com Abdurrahman Kilimci akilimci@cu.edu.tr Abstract Translation courses are a vital part of undergraduate English Language Teaching (ELT) programs and the importance of finding new ways to enhance student learning in this context cannot be stressed enough. It is reported that second language (L2) learners of English tend to produce incorrect or deviant collocations in their L2 written outputs, be it their academic or casual writing or translation products due to failure to recognize them as expressions to be learnt. In this regard, this study sought to implement the Data-driven Learning (DDL) approach in the ELT translation course to raise L2 learners’ consciousness of verb-noun collocations and assess the effectiveness of the approach using a pre-experimental pre- test/post-test design and a survey to evaluate the effectiveness of the instruction. In the study, 16 participants (13 females and 3 males) completed a six-week program. The results obtained from the Wilcoxon signed-rank test applied to compare the mean ranks of the learners’ pre- test and post-test scores indicated a significant improvement in the collocational knowledge of the targeted expressions from Pre-test to Post-test 1 (Z = -3.519, p = .005). Survey results indicated that the majority of the students found the corpus application in the translation course beneficial as a pedagogical resource with the exception of a few students, who stated that they experienced difficulties due to unfamiliar vocabulary and limited number of examples in the collocate output. Keywords: collocation, data-driven learning, concordance, consciousness-raising, translation 1. Introduction Huang (2001) reports that ELT learners are prone to producing incorrect or deviant expressions instead of appropriate collocations, when translating from first language (L1) into second language (L2). Many collocation errors by L2 learners are attributed to L1 influence, and hence are “interlingual (Laufer & Waldman, 2011). Similarly, according to Nesselhaulf (2005), about 50% of errors made in the context of collocation use by L2 learners exhibit L1 influence, which is related to language switches and blends. Laufer and Waldman (2011, p. 654) identify the cause as likely to be due to “confusing one of the collocation components with a semantically related word and consequently combining it with the collocate of the confused word”. It is therefore possible for L1 learners to make L2 translation mistakes that are completely irrelevant with their mother tongue. All and all, although it is expected that rigorous training and new classroom methods may contribute to reducing learners’ mistakes, the findings discussed by Laufer and Waldman (2011, p. 654) pertaining to the errors in collocation use of advanced L2 learners suggest that “the mailto:asliakkoyunlu@gmail.com mailto:akilimci@cu.edu.tr Akkoyunlu & Kilimci 370 acquisition of collocations lags behind many other areas of L2 acquisition, to the extent that in many cases collocation errors may appear to become fossilized”. Although studies that are based on translator training using electronic corpora are limited, researchers have drawn important conclusions, examining the effects of teaching formulaic expressions to English as a foreign language (EFL) or English as a second language (ESL) learners upon their L2 written output in the casual (Pfeiffer, 2014) and academic (Al-Hassan & Wood, 2015) sense. According to the results of these studies, associating idioms and similar collocations with their counterparts in an EFL learner’s native language improves their fluency in written English, helping them to produce native-like texts (Pfeiffer, 2012). This is a quality that is most often lacking in translator candidates and its solution could apply to this domain as well. In this respect, several studies (Chan & Liou, 2005; Koosha & Jafarpour, 2006) show that increasing learner consciousness of linguistic patterns through classroom activities based on Data-driven Learning (DDL) (Johns, 1991) adds to L2 learners’ collocational competence. However, there are certain points that need to be addressed: (a) most studies lack specific details in the context of instructional design as far as classroom activities based on DDL are concerned (b) studies report that direct access to corpora as part of DDL pedagogies may be dependent on technical prowess of learners such as being able to carry out computer database queries (c) there are not many studies regarding the use of the DDL approach in a translation course, and where they do, the participants of the study are not prospective ELT teachers but translator candidates. In order to address these problems, this study aims to enhance the prospective ELT teachers’ knowledge of lexical collocations through Consciousness-raising (C-R) (Smith, 1981) activities utilizing the DDL methodology and showcase in detail the chain of decisions made in designing such a classroom practice. The study specifically focuses on verb-noun collocations since such phrases present difficulties for L2 learners, as is attested by studies using such elicitation techniques as translations and cloze tasks (Bahns & Eldaw, 1993) and corpus analysis (Laufer & Waldman, 2011; Nesselhauf, 2005). 2. Background Although initially considered as a methodological basis rather than a domain of research, it is now widely accepted that Corpus Linguistics (CL) goes beyond this role and is regarded as a “new way of thinking about language” (Bonelli, 2000, p. 205). CL also has got its own unique approach to defining language units, in that, it has been claimed that a space between letters may not necessarily be regarded as a “delimitation of a semantic unit” (Almela & Sanchez, 2009, p. 22). A quick review of current literature in CL reveals that collocations remain a highly popular and relevant field of study (Gries, 2013). It is reported that collocations also receive great interest in the domain of EFL and ESL teaching in the recent years as their use is claimed to enable the speaker to display higher fluency in linguistic output due to their ubiquity in language and hence promote some sort of motivation in the students (Peters, 2014). Shin and Nation (2008) claim that, in order for an EFL learner to achieve native-like fluency in the language, she has to have an equally rich repository of collocations in her mind as compared to a native speaker of the language. Described as “a composite unit which permits the substitutability of items for at least of one of its constituent elements” (Cowie, 1981, p. 225), collocations are linguistic multi-word units that appear to follow certain formulae, hence belong under the umbrella term of formulaic expressions in corpus linguistics, and that help enhance learner vocabulary use International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 371 (Farghal & Obiedat, 1995). According to Kecskes (2007), formulaic expressions include but are not limited to fixed expressions, lexical metaphors, idioms, situation-bound utterances and last but not least, collocations. The theory of a functional formulaic continuum set forth in this work of Kecskes categorizes formulaic expressions based on the increasing gap between compositional meaning and actual situational meaning; with mere grammatical units such as “have to” or “be going to” encountered at one end of the spectrum and idioms with gestalt meanings far from what their individual words confer, such as “kick the bucket” or “spill the beans” being at the other. And perhaps it is from this perspective that the distinction between regular formulaic expressions and collocations be clearly observed, since collocations seem to find their place at the end that is closer to grammatical units in this spectrum. Wray (2005) further illustrates the distinction by stating that although all formulaic expressions are fixed either in part or as a whole, collocations show internal stability only to a lesser extent compared to others, being more “fluid” than the rest. Bahns (1993, p. 57) also defines collocations as “loosely fixed combinations” that lie somewhere between free- combinations, which consist of words that are entirely interchangeable in their context, and idioms, which do not offer anything in the way of interchangeability of word elements. This means that words in pairings of collocations are more subject to change as per speaker tendencies and preferences. The present study specifically focuses on the verb-noun collocations. These types of collocations have been investigated in a number of studies and have been found to pose problems for L2 learners such as the use of inappropriate synonyms, incorrect L1 translations, inappropriate collocations and underuse or no use of the expected collocation (Bahns & Eldaw, 1993; Nesselhauf, 2005; Laufer & Waldman, 2011; Can, 2017). The reason why L2 learners tend to produce inappropriate collocations is attributed to the mostly transparent nature of these expressions that leads to learners’ failure to recognize them worth learning. In this regard, learners need to be made aware of the fact that “there are combinations that are neither freely combinable nor largely opaque and fixed (such as idioms) but that are nevertheless arbitrary to some degree and therefore have to be learnt” (Nesselhauf, 2005, p. 252). Thus, C-R is one of the methods proposed by researchers in order to improve learners’ collocation use (Willis & Willis, 1996; Ying & Hendricks, 2004; Mahvelati & Mukundan, 2012; Nesselhauf, 2003). C-R actually refers to the principle of actively involving the learner to seek out language regularities in text or speech (Willis & Willis, 1996) and it is a method generally employed for reinforcing learner proficiency. It is defined by the same researchers as “activities which encourage [students] to think about samples of language and to draw their own conclusions about how the language works” (p. 63). The linguistic performance of students that are supported by C-R activities are usually related to grammar use, and yet, the measurement of this performance need not be limited to a singular scope. For instance, O’Brien (2015) has evaluated the grammar-use performance of 30 ELT students from UAE that have been subjected to C-R methods over error-correction and proofreading activities and achieved positive results. Studies indicate that C-R can be promoted through pedagogical applications of corpora (Chan & Liou, 2005, Chen, 2011; Daskalovska, 2015) in the form of DDL, which Johns (1991) define as an approach to learning that perceives a language learner as “a research worker whose learning needs to be driven by access to linguistic data” (p. 2) by means of corpora and a concordance program which generates a concordance of a text or corpus, i.e. “a collection of all the contexts in which a word or phrase occurs in a particular text or corpus of texts” (Johns, 1994, p. 319). In DDL, in some cases, the content is derived by the course instructor and sensibly turned into exercise handouts and such, while in others the Akkoyunlu & Kilimci 372 student directly uses the concordancer as a point of reference and without much instructor interference, not unlike a dictionary (Johns, 1986). Thus, according to Lenko-Szymanska and Boulton (2015), there are two main modes for using these grand databases as educational tools in the linguistic context, referred to as direct or indirect exposure to corpora. Direct exposure to corpora involves the removal of the teacher as a solid layer (and perhaps a strong filter) between learner and corpus content and is a method where the student directly accesses corpus and conducts searches on the concordancer via a computerized interface. This has been the actual, initially envisioned model for DDL. It is therefore regarded in the domain of Computer Assisted Language Learning (CALL) (Cobb, 1997). Whereas direct exposure to corpora through classroom concordancing requires learners to have stronger technical skills for conducting database queries (Lenko-Szymanska, 2015), researchers have proven that indirect approaches, such as classroom activities involving paper-based concordance exercises derived from corpus are effective in increasing student success in ELT courses (Boulton, 2010). As far as translation courses are concerned, DDL practices, such as classroom activities of concordance based on multilingual corpora have been reported to increase learner motivation in translation courses (Ulrych, 2002). However, Hu (2016) argues that in recent years, the study of translation teaching has lagged behind studies on translation theory, which, in turn, lagged behind studies on translation practice. Still, corpus tools have found their way into translation teaching, too. The initial suggested method of employing corpora in translation courses has been to use bilingual, parallel corpora (Zanettin, 1998). Similarly, categorizing the corpora available for use by translators as (a) monolingual (single and comparable) corpora, (b) parallel corpora and (c) bilingual / multilingual corpora, Kenny (2014) views corpora as valuable tools in translation studies. Bowker (2001), on the other hand, suggests a different use of corpora in translator training. Stating that “translation evaluation is highly problematic because of its subjective nature” (p. 347), she recommends the use of corpora for objective evaluation of student translation performance, as it “provid[es] reference for a teacher to verify their intuition about linguistic expression and offer[s] convincing evidence regarding the assessment of the quality of translated texts”. As laid out by Malmkjær (2004), besides acting as “translator aid”, corpora may have an important role to play in translator training by being a source of learning activities and of knowledge about the language. Hu (2016) therefore states that, the use of corpus tools in translator training (beyond “student evaluation”) involves rigorous work including the designing of a syllabus, relevant classroom activities, teaching and assessment methodologies, and exercises. It is also important to design the course in such a way that students are encouraged to engage in the corpus-based activities as much as possible. Hence, Hu (2016) advises that students that make use of data-driven, corpus-based approaches in a translation training need to have their classroom roles shifted from passive to active. This requires a complete overhaul of the existing pedagogies, textbooks, and syllabi for a given course. New exercises need to be made, and students should also be provided “with sufficient time for investigating specific translation topics, such as a corpus-based study of the English translations of culture-loaded words” (p. 184). In conjunction with this point of view, Hu suggests that, as far as translator training is concerned, using a monolingual corpus for gaining insight on the use of certain words or syntactic structures in the target language may be beneficial. The translator trainee may consult the corpus concordancer for the translation of a certain lexicon, and more effectively search for the meaning and usage of a particular word or phrase. Most of the works reviewed thus far focus either on translation studies or on its sub- International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 373 branch of translator training. Although there exist numerous studies in the domain of CL that deal with the use of corpora (a) in translation studies (b) in the education of translator trainees, it can be said that “the relevance of corpus-based translation activities in second language learning settings has been explored to a lesser extent” (Zanettin, 2009, p. 209), which suggests that a gap exists in the literature as far as corpus-based data-driven approaches in translation courses targeting EFL students are concerned. This claim may be even more accurate when the L2 learners in question assume the particular role of prospective English language teachers. It is this existing research gap that the present study aims to contribute to. 3. The Study 3.1 Aim The study involves (a) the design of DDL activities for raising learners’ consciousness of verb-noun collocations (b) a pre-test post-test single group pre-experiment that seeks to test the following hypothesis: H1: The prospective ELT teacher students, after they have received DDL instruction through consciousness raising activities, shall display greater success at using verb-noun collocations in their L1 to L2 translations than they do at the beginning of the course. In order to elaborate the findings and identify best practices and pitfalls for the research designed, the second stage of the study involves administering a questionnaire to the students and analyzing the data through quantitative methods. This practice seeks to answer the following research question: RQ1: What are the strong and weak points of using the DDL approach in designing a C-R pedagogy on verb-noun collocations in a translation course? 3.2. Participants The study group consisted of 16 third-year students (13 females and 3 males) taking up the undergraduate course of English to Turkish Translation at the ELT department at a university in Turkey during the Spring semester of the 2015-2016 Academic Year. None of the students had a previous experience of a DDL-based C-R pedagogy. The participants were not administered a test concerning the language proficiency and computer skills. They were assumed to have a similar level of proficiency in English language as they were in their third year of study and had already completed the pre-requisite course entitled “Translation: English-Turkish”. The participants were also assumed to possess enough basic computer skills to follow the three-hour training session on DDL. 3.3. Corpus For the purpose of selecting a corpus for concordance activities in the context of data- driven learning, the academic literature was referred to. Following Leech et al. (2001), who point out that the British National Corpus (BNC) is a “finite, balanced and sampled corpus” (p. 1), which contains linguistic elements with a ratio of 90% derived from written and 10% from spoken English in the form of conversations, novels and news reports, the BNC was selected in this study as the tool used for direct and indirect exposure tasks. The balanced distribution of elements, the hundred-million size, its content derived from present-day English (no earlier than 1960) and the various statistical helpers make the BNC a well- rounded, all-purpose corpus tool. Finally, the BNC currently has an active and maintained online user interface accessible by the students, teachers and researchers. Akkoyunlu & Kilimci 374 3.4. Instruments 3.4.1. Pre-test and post-test The pre-test and post-test were based on the Turkish to English translation tests, each of which consisted of 100 different Turkish sentences, each requiring the use of one English verb-noun collocation in the English translation. The tests didn’t include the sentences used for Turkish–English translation tasks during the instructional phase. The set of verb-noun collocations associated with the test items were selected from the BNC. Nesselhauf (2005, p. 256) advises that the selection of the verbs and their collocations should follow three criteria with equal weight: frequency, degree of difficulty or susceptibility to deviation, and disruption. The selection criteria adopted in the study was mainly based on the frequency. However, the criteria ‘degree of difficulty or susceptibility to deviation, and disruption’ was also taken into consideration in the construction of the tests. To construct the tests, first, a frequency list of verbs in the corpus was generated using the headword or lemma frequency functionality of the BNCweb. After the 10 most frequent verbs were identified, a ranked list of 10 most frequent noun collocates for each verb was compiled, depending on the Log- likelihood statistical metric (see Table 1). Finally, each test contained 100 items with each item corresponding to one verb-noun collocation in the 10x10 set and measuring the correct use in Turkish to English translation of the verb-noun collocations in question. Statistical tests showed that the pre-test and the post-test yielded a Cronbach’s Alpha Reliability Coefficient of r = 0.924 and r = 0.911, respectively. The pre-test is provided in Appendix A. The tests were administered to students on paper. Table 1. Verb-noun collocations used in Pre-test and post-test and DDL activities Set Associated Verb (headword frequency) Associated Noun Combinations for the Verb (and their headword frequencies in combination with the verb, reported by the BNC) 1 Have (1316636) problems (155.0601), reputation (191.0513), difficulty (541.0377), chance (628.0058), effect (919.4065), doubts (184.3252), advantage (245.1934), experience (189.0276), idea (509.9554), access (200.8409) 2 Do (537577) harm (1090.3479), damage (475.2488), housework (200.2118), washing (84.3233), crossword (59.8033), ironing (47.4099), cooking (45.6653), deal (39.6628), favour (34.9621), calculations (17.6712) 3 Get (213376) Chance (1046.4567), job (3017.5883), impression (863.1962), permission (270.7638), glimpse (16.9528), revenge (59.7266), allowance (44.6136), refund (43.6929), benefit (39.0464), passport (27.5591) 4 Make (210266) decision (9351.9706), sense (9351.9706), difference (7451.0629), mistake (6970.4482), progress (6039.2723), contribution (5887.544), effort (4546.7912), attempt (4025.2815), arrangements (3651.8692), statement (3083.2802) 5 Know (178223) things (327.5725), truth (323.0295), answer (266.4245), facts (105.8882), secret (39.2278), technique (13.3392), details (11.9355), password (9.1265), tricks (6.3705), meaning (2.6296) 6 Take (173609) advantage (11194.7948), action (9036.0304), part (8820.4581), care (8526.9772), photographs (1391.3866), steps (6801.7822), breath (2698.8826), responsibility (2499.5376), precautions (1550.4181), risks (1478.5243) International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 375 7 Give (126193) Impression (5574.9), chance (4276.3341), advice (4001.5705), opportunity (3553.1153), priority (3027.6287), birth (2896.4865), evidence (2521.4191), details (2357.4061), information (2346.2823), instructions (1377.1996) 8 Use (105744) language (1976.5704), data (1375.2459), information (670.7136), program (621.6381), technology (587.8285), terminology (443.2755), tactics (377.312), weapon (340.299), symbols (313.9541), facilities (275.0893) 9 Find (95621) way (3148.2298), solution (783.0322), evidence (414.4322), place (305.8166), keyword (213.627), accommodation (210.7156), answers (196.8028), fault (174.8683), examples (173.7614), hope (166.4034) 10 Put (67694) pressure (2294.6116), money (1008.5519), emphasis (372.1679), bandage (82.3028), question (343.8783), end (196.0513), brakes (111.1171), blame (191.1317), name (103.1352), idea (101.6366) 3.4.2. Questionnaire In order to diagnose the strong and weak points of the design, the evaluations of the students regarding the experience they went through while interacting with the BNC through direct concordance activities had to be collected and interpreted as data. For this purpose, a 32-item six-point scale consisting of ‘strongly disagree/somewhat disagree/disagree’ and ‘somewhat agree/agree/strongly agree’ was constructed. The questionnaire was mainly intended to evaluate the students’ perceptions of and attitude to the corpus use in the L1-L2 translation course in order to determine the effectiveness of the DDL instruction on the students’ verb-noun collocation use and their translation competence. In developing the questionnaire, the researchers drew on Yoon and Hirvela (2004), who developed a measurement tool for evaluating the East Asian students’ attitude towards the use of a corpus concordancer within the context of an L2 academic writing course. Yoon and Hirvela’s questionnaire, which is comprised of 42 Likert-type items in the 1-6 scale, is based on the use of Collins COBUILD and is of a Cronbach’s Alpha Reliability Coefficient of r = 0.96. The 42-items of this questionnaire were modified and adapted by the researchers for the specific case of BNCweb use and verb-noun collocation use. After running a reliability analysis for this new version, items that reduced test reliability were removed and the remaining 32-items that displayed a Cronbach’s Alpha coefficient of r = 0.89 were included in the new scale, named the Questionnaire about using the BNC in ESL Translation. The survey presented in Appendix B, was administered to students over a computer interface using Google Forms. 3.5. Procedure The study was conducted in the context of the Turkish-English translation course of the ELT program at the Faculty of Education at a university in Turkey during the 2015-2016 Spring semester. There are actually two “Translation” themed courses in the undergraduate curriculum of the ELT program, with the other being a 14-week long, compulsory course taken at the winter semester of the 2nd year, titled “Translation: English-Turkish”. The said course aims to improve translation skills of students from L2 into L1unlike the former, which focuses on translation from L1 into L2. The reason for choosing the Turkish-English variant of the translation courses over the former one was motivated by the view that, as far as EFL education is concerned, collocations are designated as an important linguistic element to master the use of, for generating linguistic output akin to native users of the language. (Pfeiffer, 2012; Wood, 2009; Shin & Nation, 2008). The program based on a 6-week long intervention was designed to be carried out within a formal education setting at Akkoyunlu & Kilimci 376 undergraduate level, where face-to-face instruction may take place, i.e. a classroom. The target course for the program, Turkish-English Translation, is a 14-week long, compulsory one taken during the Spring semester of the 3rd year at the ELT undergraduate program. The study followed a pre-test/post-test pre-experimental design, which was comprised of two phases: (a) design phase (b) empirical phase. The design phase was to design the instructional program by analyzing the current situation and seeking help from existing literature in the field and find ways to implement a data-driven pedagogical approach for consciousness-raising to improve collocational use of advanced L2 learners in a translation course. During this phase, paper-based and hands-on concordance activities were designed. These, as well as other teaching materials were derived from corpora due to convenience purposes. The empirical phase, which followed a quantitative paradigm of research, consisted of a pre-test, instructional treatment, a post-test and a survey instrument, Questionnaire about using the BNC in ESL Translation. The phase involved carrying out a pre-experiment for the purpose of evaluating the effectiveness of the program designed in phase one. The independent variable for the pre-experiment was designated as an intervention defined as: “receiving a translation course through data-driven pedagogical methods for raising consciousness towards the use of the most frequent verb-noun collocations in the English language”, whereas the dependent variable was designated as academic success in appropriately using the most frequent verb-noun collocations in Turkish-English translations. Prior to the instructional intervention, the participants were given a three-hour training on DDL to familiarize them with corpus search and techniques in order for them to take part in the corpus-based classroom and concordancing homework activities throughout the course. Two tests were administered to the participants to determine the effect of the data-driven pedagogical approach employed to raise their consciousness towards the appropriate use of verb-noun collocations in the Turkish-English translation course. First, a pre-test was administered to the study group in order to determine their prior knowledge with relation to the targeted verb-noun collocations. Second, a series of classroom lectures over the course of 6 weeks were delivered to the students. Alongside the regular course content, the course targeting the students in the study group focused each week on the use of collocations through consciousness raising tasks based on data-driven methods in the form of paper and computer based concordance exercises applied each week. The instructional approach was inspired by the works of Lenko-Szymanska and Boulton (2015) and Boulton (2010) in order to effectively combine the strengths of the direct instruction, and direct and indirect exposure of students to the corpus. To this end, the following classroom and homework activities were planned out: a) Direct Instruction: Starting from day one of the intervention, students were instructed directly on the use of collocations that were within the scope of the program. This constituted the consciousness-raising element and it was made sure students were not left without guidance and prior knowledge when undertaking the concordance tasks that followed. b) Direct Exposure (Concordance Homework Assignments): After a brief initial classroom introduction to using the BNCweb query tool via an overhead projector, students were asked at the end of each course to carry out homework assignments which they could finish only by using the BNC on their own, outside of the classroom. Leaving the concordancing task outside of the classroom was not only a resource-efficient decision, eliminating the need for occupying the school International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 377 computer laboratories but also promoting learner autonomy and thereby encouraging discovery learning. The homework assignments at the end of each week provided students with a certain verb and required them to use the BNC to find 10 sentences that included a verb-noun collocation that contains the given verb. It was also required that each of the 10 sentences featured a different noun in combination with the given word. In the end, the students were required to translate the collections of sentences they derived from the corpus in this manner. This way, 2 verbs were given to students at the end of each week for a total of 10 verbs at the end of 5 weeks, and students translated from English into Turkish a total of 100 sentences including verb-noun collocations of 10 most popular verbs in English language. c) Indirect Exposure (Corpus-based Classroom Activities): During each classroom lesson, students have been handed out exercise sheets which were comprised of fill-in-the-blanks style questions created from BNC-derived text. A sheet for each of the selected 10 verbs, each of which contained 10 fill-in-the-blanks questions was handed out to students each week for a total of 10 sheets and 100 questions done at the end of 5 weeks. Each question in each sheet was associated to one of the top-ten most frequently used nouns within collocations of the verb. Also, after an initial attempt at raising awareness of students towards collocations in the first week, a single 100-item exercise sheet was handed to students. This sheet also derived from the BNC featured questions similar to the previously explained 10 sheets, but was different from them in that it required students to fill in the gaps with appropriate verbs (instead of nouns) to complete a verb-noun collection correctly. A sample of these sheets is shared in Appendix C. The indirect exposure to dedicate classroom hours to working on paper-based handouts was considered for the following pedagogical, practical and technical reasons: a) Direct exposure to corpora requires access to a computer laboratory with internet connectivity. The setting where the study took place had a computer laboratory, albeit with an overloaded schedule. Therefore, it was thought that the classroom concordancing time would be severely limited. b) Also, the limited amount of time allocated for classroom concordancing could be overloaded with student questions that could rise from technical problems, leaving an even narrower time frame for actual concordance activities. c) Today, it can be safely assumed that all students have an access to a personal computer in their homes. Therefore, direct exposure tasks may be assigned to students as homework to be carried out using their personal computers, in their own time and in the comfort zone provided by using a device they own and are familiar with (unlike a laboratory computer). And concordancing as a homework assignment could leave students to discover through trial and error the use of the corpus digital interface, taking as much time as they need. At the end of the instructional intervention, students were administered a post-test to compare statistically the average academic success at the appropriate use of verb-noun collocations in Turkish-English translation with the measurement at the beginning of the intervention, the pre-test. The last instrument utilized in the study was the DDL Instruction Evaluation Scale, which aimed to evaluate the effectiveness of the DDL instruction and to provide insights as to what could be done to improve the instruction in terms of possible weaknesses, strengths and needs. Akkoyunlu & Kilimci 378 3.6. Data Analysis The empirical phase followed a quantitative paradigm of research and involved a pre- test/post-test single group quasi-experiment. Analysis of the quantitative data pertaining to the empirical phase of the research at hand was conducted using the SPSS (v. 21.0). The DDL Instruction Evaluation Scale was administered to the students to assess their experience regarding the use of corpus data to search and retrieve verb-noun collocations once after the intervention was concluded. For the purpose of ease of interpretation and comparison, the items in the scale were categorized thematically and discussed by the number of the participants who rated each item rather than the percentages of the respondents in order not to present misleading information to the reader as relatively few number of students participated in the questionnaire. 4. Results 4.1. Experimental Treatment Results The two sets of data from the pre- and post-tests tests aiming to investigate the effects of the DDL activities on learners’ Turkish-English translation performance were scored and mean scores were calculated. The descriptive statistics for these tests is given in Table 2. Table 2. Descriptive statistics for the pre-test and the post-test for verb-noun collocations N Min. Max. Mean SD Pre- test 16 17 67 50.25 13.19 Post- test 16 65 100 91.44 8.22 The preliminary analysis revealed that the range of distribution were different among the pre-test and the post-test with the minimum and maximum values for the pre-test being 17 and 67 and for the post-test 67 and 100, respectively. And there was a large difference between the mean scores, with the pre-test averaging at 50.25 (SD = 13.19) and post-test at 91.44 (SD = 8.22) as is presented in Table 2. In order to assess whether there was a statistically significant change in test scores before and after the intervention, a paired samples t-test was considered to compute the difference between two related data. However, a non-parametric equivalent, Wilcoxon signed-rank test was decided on due to the following reasons; (a) the sample size (N=16) was relatively low (b) visual examinations of histogram charts, box-plot graphs, as well as Shapiro-Wilk tests revealed that the sets of data (both pre-test and post-test) had non-normal distributions, which violated the assumption of normality associated with the parametric Paired Samples t-test. Conclusively, a non-parametric statistical test, the Wilcoxon signed-ranked test was used to compare the pretest and posttest scores of the same sample. The results of the analysis are shared in Table 3. International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 379 Table 3. Wilcoxon signed-rank test results comparing the scores of the pre-and post-tests. Tests N Mean Rank Sum of Ranks z p Post Pre Negative Ranks 0 .00 .00 -3.519 a .0005 Positive Ranks 16 8.50 136.00 Ties 0 a: based on negative ranks Analysis according to the Wilcoxon signed-rank test indicated a statistically significant difference in the students’ performance from the pre-test to the post-test (Z = -3.519, p < .0005). 4.2. Results of the Questionnaire about Using the BNC in ESL Translation This section presents the results obtained from the 32-item Likert-type questionnaire administered to the 16 students after the completion of the 6-week DDL instruction. The questionnaire aimed to learn the students’ perceptions of and attitudes to the DDL instruction in terms of its strengths and weaknesses. The results are presented by categories and discussed by the number of the participants. Figure 1 presents the results of the category of items aimed at determining whether the students encountered any problems or difficulties when using the corpus. This category is mainly comprised of negatively disposed items, with the exception of the third item. Most of the students (n=14) point out that the search facility in the BNCweb didn’t pose any difficulty for them. The top two items are the ones where student responses display an uneven distribution. While 10 students state that they didn’t experience any difficulty due to the unknown words and the limited number of sentences in the concordance and the collocation outputs, 6 students report that these issues presented difficulties to them. Other two items - items 6 and 9-, which indicate that some students had some difficulty in using corpus are related to the high number of sentences in the concordance output and the text difficulty. One-fourth of the students (n=4 for each item) expressed that they had difficulties due to these issues, while the other students (n=12 for each item) pointed out that these factors didn’t present any difficulties for them. Overall, the majority of the students stated that they didn’t experience any difficulties in using the corpus and in analyzing the concordance and collocation outputs they generated with regard to the cut-off sentences, time limitation, the effort exerted and the text difficulty during the DDL instruction. Akkoyunlu & Kilimci 380 Figure 1. Problems and difficulties that students encountered in corpus use (n=16) 16 12 8 4 0 4 8 12 16 The real texts in the corpus were too difficult to understand I had some difficulty in analysing concordance output I had some difficulty in analyzing collocate output I had some difficulty in using the corpus due to too many sentences in concordance output I had some difficulty in using the corpus due to time and effort spent on analyzing the data I had some difficulty in using the corpus due to cut-off sentences in concordance output The searching technique was easy to learn. I had some difficulty in using the corpus due to the limited number of sentences in the concordance output I had some difficulty in using the corpus due to unfamiliar vocabulary on concordance/collocate output Strongly Disagree Somewhat Disagree Disagree Somewhat Agree Agree Strongly Agree International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 381 Figure 2. Student responses regarding the instructor's assistance and paper-based approach (n=16) Figure 2 presents the results gathered from the items which were intended to find out to what extent the instructor’s intervention to assist the students during their corpus analysis was beneficial and to what extent the students could conduct corpus analysis on their own. All the students (n=16) noted that they found the practice sessions useful intended to teach them the DDL technique (item 4). On the other hand, almost half of the students (n=7) responding to item 1 related to the necessity of the instructor’s assistance for determining the meanings of collocations indicated that that they could not have found out the meanings of collocations without the instructor’s help. In contrast, a relatively high number of students (n=5), considering the total number of participants (n=16) in the survey, disagreed that they could have extracted the collocations from the concordance output without the instructors’ assistance. Regarding the rest of the items related to the use of paper-based concordances and the instructor’s guidance, almost all the students found the paper-based concordances and the instructor’s guidance helpful when identifying collocations and conducting corpus analysis. 16 12 8 4 0 4 8 12 16 Paper-based handouts of selected concordance lines were useful for me to capture the additional meanings of collocations. I could have easily found the collocations in the concordance output without the instructor's help. The practice sessions were helpful for learning the technique. Paper-based handouts of selected concordance lines are necessary to save time. The instructor’s guidance was useful for me to find the different meanings of the collocations in concordance output. I could have easily found out the meanings of the collocations in the concordance output without the instructor's help. Strongly Disagree Somewhat Disagree Disagree Somewhat Agree Agree Strongly Agree Akkoyunlu & Kilimci 382 Figure 3 represents the students’ views on whether they found the application of corpus analysis into the translation course beneficial for both increasing their collocational knowledge and improving their translation skills and whether they thought that they would continue using corpus in their translation practices and other courses in the future. All the participants (n=16) rated items 2, 3, 8, 12 and 14 pertaining to the benefit of corpus use for learning collocations as agreed to strongly agreed. It is noteworthy that most of the students expressed a strong agreement on these statements. Regarding the statements (items 6 and 11) about whether the students would use by their own choice and whether their translation skills would have been better if they had known the corpus earlier, all (n=13) but three (who responded as ‘somewhat disagree’ to ‘disagree’) agreed that they would continue utilizing corpus in their language learning and they would have been better at translation than they were then. Overall, the students agreed that corpus use supported their translation skills and improved their knowledge of verb-noun collocations. Finally, Figure 4 presents the students’ overall evaluation of and attitude to the corpus use in the DDL instruction. All the students but one (who somewhat disagreed on the recommendation of the corpus to other students) generally strongly agreed that the corpus was a useful source in translation courses and that they would not only recommend using this resource in the translation course but also recommend it to other students in their department. International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 383 Figure 3. Students’ perceptions of and attitudes to corpus pedagogy in translation course 16 12 8 4 0 4 8 12 16 Using the corpus is helpful for learning to translate The corpus is more helpful than a dictionary to improve translation skills. Using the corpus is helpful for learning the meaning of collocations If I had known the corpus earlier, my translation skills would have been better When I am unsure about the usage of a collocation, I will search for it in the corpus. When I am unsure about the meaning of a collocation, I will search for it in the corpus. Corpus use should be taught in English classes. I will use the corpus for my translation practices in the future. I will often use the corpus by my own choice. The corpus should be introduced in all ESL translation courses. As I have learned more about the corpus, I have come to like it more. I understand the purpose of using the corpus in this course. Using the corpus is helpful for learning the usage of collocations. I want to use the corpus in my other ESL courses too. Strongly Disagree Somewhat Disagree Disagree Somewhat Agree Agree Strongly Agree Akkoyunlu & Kilimci 384 Figure 4. Students' overall evaluation of corpus use in translation learning 5. Discussion The results from the comparison of scores obtained from the pre-test and the post-test on verb-noun collocation indicated that there was a statistically significant increase in the students’ knowledge of verb-noun collocations, which suggests that the DDL instruction applied to the Turkish to English translation course was successful in achieving its objectives. Hence, the research hypothesis, which state that “the prospective ELT teacher students, after they have received DDL instruction through consciousness raising activities, shall display greater success at using verb-noun collocations in their L1 to L2 translations than they do at the beginning of the course” was confirmed. This finding is in line with the findings of Chan and Liou (2005), who worked with 32 college students in Taiwan and found out that concordancing activities, albeit with a bilingual web-interface, supported with fill-in- the-blanks and translation style exercises significantly improved the correct use of verb-noun collocations. Hence, their claim that “an eclectic approach that combines concordancing and other traditional instructional methods may help learners better than the approach that relies on a single dominant teaching method” (p.248) was consolidated with the present study that employed both direct and indirect corpus use. Apart from supporting Chan and Liou’s findings, a contribution of this study to the existing literature in the field can be the notion that students can successfully work with a monolingual concordancer in their translation studies as well, and that they can autonomously carry out concordancing tasks at home and in their own pace, leaving the valuable classroom time for other learning activities. As for the research question specified at the outset of the study, the analysis of the responses to the questionnaire revealed that the students generally found corpus concordancing to be a beneficial activity, both for generic English courses and for translation in particular. The findings indicated that the students highly benefitted from the corpus-based approach implemented in the Turkish-English translation course without much difficulty. For instance, the low responses to the negatively disposed items (Figure 1) were interpreted that students did not experience much difficulty in working with the corpus data and tool. These findings are in parallel with Chambers (2007), who state that learners show positive reactions to corpus-based activities despite a number of important obstacles observed. On the other 16 12 8 4 0 4 8 12 16 I recommend using the corpus in the same course in the future. I will recommend the corpus to other students in my department or elsewhere. Overall, the corpus is a very useful resource for translation courses. Strongly Disagree Somewhat Disagree Disagree Somewhat Agree Agree Strongly Agree International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 385 hand, unfamiliar vocabulary and limited number of examples in the collocate output stood out as the most prominent downsides of the corpus use (Figure 1). The reason for the students’ somewhat agreeing/agreeing with the relevant item asking whether they had any difficulty using the corpus due to unfamiliar vocabulary in the questionnaire could be explained by the fact that he richness of vocabulary in corpus concordance output may be intimidating to L2 students, particularly those that do not have a rich repository of words at their disposal. The limitation in the number of sentences in the concordancing output was also referred to, albeit slightly, as a problem. This is a questionable finding, as the tasks required students to search the corpus output for the most frequent verb-noun collocations, each of which was based on the most frequent headword verb and its most frequent noun collocate. Hence, a quick search for these expressions would reveal, at worst, hundreds of examples. The fact that this negatively disposed item ranked among the top three was an inexplicable phenomenon. Another point is that some students didn’t agree with some of the positively disposed items in the scale (Figure 2, particularly items 1 and 5), which are related to the instructor’s help in working out the meanings of collocations in the collocate output and finding collocations in concordance output. Almost half of the students (n=7 and n=5, respectively) expressed their disagreement with these items, in a way emphasizing the facilitating role of the instructor in the DDL instruction. This can be interpreted from the perspectives that the rule-based deductive approach in designing corpus-based activities is advantageous over the inductive approach, whereby learners have to infer the rules or patterns from examples (Flowerdew, 2009). In this respect, if the students hadn’t been given instruction on what procedure they should follow to determine the most frequent verb-noun collocations, they could have had a harder time both extracting the collocations and finding out their meanings from the concordance output due to “a lot of noise” in the corpus data (Flowerdew, 2009; p. 408). The last two among the least agreed upon positively disposed items were related to whether the students would use by their own choice and whether their translation skills would have been better if they had known the corpus earlier (Figure 3, items 6 and 11). Partial agreement with these items suggests that some students didn’t find corpus useful in language learning. However, this finding contradicts with the fact that almost all the students uniformly agreed that corpus use was helpful in learning the usage of collocations and improving their translation skills. 6. Conclusion It was found that there was a statistically significant difference in the students’ use of verb-noun collocations in their Turkish-English translations from before the DDL instruction enhanced with C-R activities to after the instruction. All and all, it can be inferred that, although there were a few students who stated that they were not really interested in relying on it in the future, most of the students found the corpus pedagogy, hence the DDL approach, useful in their Turkish to English translation course. Also, it was understood that, among the perceived shortcomings of using corpus as a pedagogical resource in this specific context of translation, the most prominent ones were difficulties due to unfamiliar vocabulary and limited number of examples in the collocate output. This may serve to fill a gap in the literature, as far as research that makes use of data- driven approaches in translator training is concerned. Akkoyunlu & Kilimci 386 This result brings about practical implications suggesting that, as far as undergraduate level ELT courses are concerned, corpus-based DDL approaches may be used as supportive tools for enhancing student linguistic competence not only in grammar courses but also in translation courses. In this respect, using a data-driven pedagogical approach in L1 to L2 translation courses may be helpful to decrease L1 to L2 translation errors, which may result from improper use of lexical collocations, and to enrich the students’ lexical repertoire in relation to verb-noun collocations. In addition to the indirect exposure to corpora, as was the case in this study, the direct exposure to corpus may also be achieved by assigning concordancing activities to students as homework, encouraging the learner autonomy and saving classroom time. In this regard, instructors might not need to be afraid to hand over the steering wheel to the student and let him/her handle the Web-based concordancing tasks directly, as part of homework assignments on condition that the aim and the focus is well- delineated. It should be emphasized that the findings should be interpreted with caution for the following reasons. First, the limited number of participants (n=16) poses a threat to external validity of the experiment. Therefore, the results may not be generalized to a larger population. Second, due to time and resource constraints, the researchers were unable to come up with a control group with which to compare the results of the experimental group that received the intervention, hence a single-group pre-test/post-test pre-experimental design was considered in this study. It is understood that the “instrumentation effect” caused by the translation course itself may not have been accounted for in such a design of experiment. That is to say, the change in learners’ knowledge of lexical collocations may be caused by the natural progress of the translation course or by other factors and not by the specific method of instruction itself. In this respect, this is a risk to internal validity in scientific research and is another limitation of the study at hand. Considering that this study was based on activities limited to a relatively short span of 6 weeks and a small sample size of 16 participants and did not involve qualitative research methods, there probably exists a need to investigate the underlying reasons for negative attitudes and perceptions concerning corpus use in language learning in general in the long run and in a deeper or wider manner. Therefore, future efforts investigating the use of data- driven approaches in L2 translation learning should perhaps employ detailed interviews with students and scrutinize the way they perceive the use of corpus and its tools. The tasks, within the scope of this study, were also rather limited in that they involved finding sentences with the collocations specified and translating them at home, discussing the translated output in the classroom, and doing the fill-in-the blank type exercises based on the verbs and their collocates derived from the corpus by the instructor. Therefore, it may be useful if future research efforts concentrate on more creative task ideas in translation courses. Also, theories based on literature may be developed concerning how corpus-based activities may be extended in order to improve overall translation quality and not simply success at the lexical collocation use. International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 387 References Al-Hassan, L., & Wood, D. (2015). The effectiveness of focused instruction of formulaic sequences in augmenting L2 learners' academic writing skills: A quantitative research study. Journal of English for Academic Purposes, 17, 51-62. Almela, M., & Sánchez, A. (2009). Words as “lexical units” in learning/teaching vocabulary. International Journal of English Studies, 7(2), 21-40. Bahns, J., & Eldaw, M. (1993). Should we teach EFL students collocations? System, 21, 101–114. Bahns, J. (1993). Lexical collocations: a contrastive view. ELT Journal, 47(1), 56-63. Boulton, A. (2010). Data‐driven learning: Taking the computer out of the equation. Language Learning, 60(3), 534-572. Bowker, L. (2001) Towards a methodology for a corpus-based approach to translation evaluation. Meta, 46(2), 345-364. Can, C. (2017) A learner corpus-based study on verb errors of Turkish EFL learners. Journal of Education and Training Studies, 5(9), 167-175. Chambers, A. (2007). Popularising corpus consultation by language learners and teachers. In E. Hidalgo, L. Quereda, & J. Santana (Eds.), Corpora in the foreign language classroom: Selected papers from the sixth international conference on teaching and language corpora (TaLC 6) (pp.3-16). Amsterdam/New York: Rodopi. Chan, T. P., & Liou, H. C. (2005). Effects of web-based concordancing instruction on EFL students' learning of verb–noun collocations. Computer Assisted Language Learning, 18(3), 231-251. Chen, H. J. H. (2011). Developing and evaluating a web-based collocation retrieval tool for EFL students and teachers. Computer Assisted Language Learning, 24(1), 59-76. Cobb, T. (1997). Is there any measurable learning from hands-on concordancing? System, 25(3), 301-315. Cowie, A. P. (1981). The treatment of collocations and idioms in learners' dictionaries. Applied linguistics, 2, 223. Daskalovska, N. (2015). Corpus-based versus traditional learning of collocations. Computer Assisted Language Learning, 28(2), 130-144. Farghal, M., & Obiedat, H. (1995). Collocations: A neglected variable in EFL. IRAL- International Review of Applied Linguistics in Language Teaching, 33(4), 315 Flowerdew, L. (2009). Applying corpus linguistics to pedagogy: A critical evaluation. International Journal of Corpus Linguistics, 14(3), 393-417. Friedman, G. L. (2009). Learner-created lexical databases using web-based source material. ELT Journal, 63(2), 126–136. Gries, S. T. (2013). 50-something years of work on collocations: what is or should be next…. International Journal of Corpus Linguistics, 18(1), 137-166. Hu, K. (2016). Introducing corpus-based translation studies. Newyork/London: Springer. Huang, L. S. (2001). Knowledge of English collocations: An analysis of Taiwanese EFL learners. In Luke, C. and B. Rubrecht, (Eds.), Texas Papers in Foreign Language Akkoyunlu & Kilimci 388 Education: Selected Proceedings from the Texas Foreign Language Education Conference, 2001. Volume 6, n1, Fall 2001. Koosha, M., & Jafarpour, A. A. (2006). Data-driven learning and teaching collocation of prepositions: the case of Iranian EFL Students. Asian EFL Journal Quarterly, 8(4), 192–209. Johns, T. (1986). Micro-concord: A language learner's research tool. System, 14(2), 151- 162. Johns, T. (1991). Should you be persuaded: Two examples of data-driven Learning. In T. Johns and P. King (Eds.), ELR Journal 4: Classroom Concordancing (pp. 1– 16). Birmingham: CELS, The University of Birmingham. Johns, T. (1994). From printout to handout: Grammar and vocabulary teaching in the context of data-driven learning. ERL Journal, 4, 27-45 Kecskes, I. 2007. Formulaic language in English lingua franca. In I. Kecskes & L. Horn (eds.), Explorations in pragmatics: Linguistic, cognitive and intercultural aspects (pp. 191-218). Berlin & New York: de Gruyter. Kenny, D. (2014). Lexis and creativity in translation: A corpus based approach. Laufer, B., & Waldman, T. (2011). Verb‐noun collocations in second language writing: A corpus analysis of learners’ English. Language Learning, 61(2), 647-672. Leech, G., Rayson, P. & Wilson, A. (2001). Word frequencies in written and spoken English: Based on the British National Corpus. Longman: London. Leńko-Szymańska, A., & Boulton, A. (2015). Introduction: Data-driven learning in language pedagogy. In A. Leńko-Szymańska & A. Boulton (Eds.), Multiple affordances of language corpora for data-driven learning (pp. 1-14). Amsterdam: John Benjamins. Mahvelati, E. H., & Mukundan, J. (2012). The effects of input flood and consciousness- raising approach on collocation knowledge development of language learners. International Journal of Applied Linguistics and English Literature, 1(6), 182-192. Malmkjær, K. (Ed.). (2004). Translation in undergraduate degree programmes. Philadelphia, USA: John Benjamins. Nesselhauf, N. (2003). The use of collocations by advanced learners of English and some implications for teaching. Applied linguistics, 24(2), 223-242. Nesselhauf, N. (2005). Collocations in a learner corpus. Amsterdam: Benjamins. O'Brien, J. (2015). Consciousness-raising, error correction and proofreading. Journal of the Scholarship of Teaching and Learning, 15(3), 85-103. Peters, E. (2014). The effects of repetition and time of post-test administration on EFL learners’ form recall of single words and collocations. Language Teaching Research, 18(1), 75-94. Pfeiffer, K. (2014). The effect of L1 on L2 formulaic expression production (Unpublished doctoral dissertation). Bilkent University, Ankara. Sharwood Smith, M. A. (1981). Consciousness-raising and the second language learner. Applied Linguistics, 11(2), 159-168. International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 389 Shin, D., & Nation, P. (2008). Beyond single words: the most frequent collocations in spoken English. ELT Journal, 62(4), 339-348. Tognini-Bonelli, E. (2000). Corpus classroom currency. Naujoji Metodologija, 24, 205– 244. Ulrych, M. (2002). An evidence-based approach to applied translation studies. In A. Riccardi (Ed.), Translation studies: Perspectives on an emerging discipline (pp. 198-213). Cambridge: Cambridge University Press. Willis D. and Willis J. (1996). Consciousness-raising activities. In Willis D. and Willis J. (Eds.), Challenge and change in language teaching (pp. 63-76). Heinemann, Oxford. Wood, D. (2009). Effects of focused instruction of formulaic sequences on fluent expression in second language narratives: A case study. The Canadian Journal of Applied Linguistics, 12(1), 39. Ying, Y., & Hendricks, A. (2004). Collocation awareness in the writing process. Reflections of English Language Teaching, 3, 51-78. Yoon, H., & Hirvela, A. (2004). ESL student attitudes toward corpus use in L2 writing. Journal of Second Language Writing, 13(4), 257-283. Zanettin, F. (1998). Bilingual comparable corpora and the training of translators. Meta, 43(4), 616-630. Zanettin, F. (2009). Corpus-based translation activities for language learners. The Interpreter and Translator Trainer, 3(2), 209-224. ENDNOTES * This paper is partially based on a thesis prepared by the first author and supervised by the second author to be submitted to the Social Sciences Institute of Çukurova University in partial fulfilment of the requirements for the MA degree in English Language Teaching. Akkoyunlu & Kilimci 390 Appendix A Pre-Test for Verb-Noun Collocation Use in Turkish to English Translation Name: Surname: Please translate the following Turkish sentences into English. 1. Bu değişkenin kullanıcı puanları üzerine etkisi bulunmakta. 2. O asla kimseye zarar vermezdi. 3. Andy Bristol’da bir fabrikada işe girdi. 4. Karar vermek zorunda. 5. Böyle şeyler bilmek mükemmel olmalı. 6. İnsanlardan faydalanmak/çıkar sağlamak için konumunu kullandı. 7. Önemsemediği izlenimini veriyor. 8. Hayvanlar dili farklı şekilde kullanırlar. 9. Bunu yapmanın bir yolunu bulmalısın. 10. Karar vermem için baskı yapıyorlar. 11. Sınavı geçme olasılığı/şansı var. 12. Yangın binaya zarar vermedi. 13. Umarım mesajı almışsındır. 14. Kararın bana mantıklı geliyor. 15. Gerçeği biliyoruz. 16. Bir an önce harekete geçmeliyiz. 17. Şirket bu stratejiye bir şans verdi. 18. Var olan datayı kullanacağız. 19. Bu probleme bir çözüm bulmalıyız. 20. Paramı eve yatırdım. 21. Yürümekte güçlük çekiyor. 22. Polis teröristlerle anlaşma yapmayı reddetti. 23. Sıkıldığı izlenimini aldım. 24. Duvarları boyamak bu odada farklılık yarattı. 25. Bu sorunun cevabını bilmiyorum. 26. Hiçbir aktivitede yer almaz. 27. Sana bir tavsiye vereceğim. 28. Öğrencilere bilgiyi nasıl kullanacakları öğretilmelidir. 29. Yakında ihtiyaç duydukları kanıtı bulacaklar. 30. Dramaya vurgu yapıyorlar. International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 391 31. Kocam ve ben okullar hakkında fikir sahibiyiz. 32. Ev işi yapmaya tahammül edemiyorum. 33. İnşaata başlamak için gereken izni aldık. 34. Testte hatalar yaptı. 35. Gerçekleri bilmek ilk önceliğimiz. 36. Şirket gelişmek için adımlar attı. 37. Program yeni şeyler öğrenme fırsatı sunuyor. 38. Hesaplamalar için bir program kullandık. 39. Uyuyacak bir yer bulacağım. 40. Sorusunu yönelttiğinde heyecanlandım. 41. Finans alanında tecrübem var. 42. Bana bir iyilik yapabilir misin? 43. Sam Lucy’yi bir an için gördü. 44. Bu yıl Fransızca’da ilerleme gösterdi. 45. Bir sır bilmek ister misin? 46. Odaya girmeden önce nefes aldı. 47. Güvenliğe öncelik vermeliyiz. 48. Teknolojiyi etkili olarak kullanmak istiyorlar. 49. Öncelikle anahtar kelimeyi bulmalısın. 50. İlişkisini sonlandıracak. 51. Bilgisayarımla ilgili sorunlar yaşıyorum. 52. Bu sabah çamaşırları ben yıkıyorum. 53. İntikamımı alacağım. 54. İşimize katkısı bulundu. 55. Tekniği biliyordu. 56. Birileri sorumluluğu üzerine almalı. 57. Kız çocuğu doğurdu. 58. Dil bilimi terminolojisini kullanıyorum. 59. Yeni öğrenciler kalacak yer bulamadı. 60. Şehir harcamalarını frenledi. 61. Sınavda diğerlerine kıyasla avantajlıydı. 62. Bulmaca çözmek için oturdu. 63. Hala annemden harçlık alıyorum. 64. Sosyal olmak için çaba harcıyordu. 65. Planın detaylarını bilmiyorum. Akkoyunlu & Kilimci 392 66. Bizim fotoğrafımızı çekti. 67. Öğretmen önce yönergeleri veriyor. 68. Maçı kazanmak için taktik uyguluyorlar. 69. Cevabı bulmak zor olmadı. 70. Kendini suçluyor. 71. Bilgisayara erişimim yok. 72. Hesaplamaları kolayca yapabiliyor. 73. Haklıysan, geri ödemeni alırsın. 74. Sınavı geçmek için girişimde bulundu. 75. Şifreyi biliyor musun? 76. Kendine iyi bak. 77. Mahkemeye kanıt sunacak. 78. Nasıl silah kullanılacağını öğreniyor. 79. Sende sürekli kusur buluyor. 80. Bebeğe isim koydular. 81. Yetenekleriyle ilgili şüphelerim var. 82. John ütü yapıyordu. 83. Bundan kim faydalanabilir? 84. Yurtdışına gitmek için planlamaları/düzenlemeleri yaptım. 85. Yaptığın bütün hileleri biliyorlar. 86. Her zaman önlemler almalısın. 87. Detayları sonra verecek. 88. Bazı kelimeler için semboller kullanıyorlar. 89. Bu tarz problemlerin örneklerini bulmak zor değil. 90. Böyle bir fikri aklına sokan ne? 91. İki otel de konuklarını sinirlendirmek açısından ün sahibi. 92. Yemekleri ben yaparım. 93. Sonunda pasaportu aldı ve Rusya’dan ayrıldı. 94. Pop starın açıklama yapması bekleniyor. 95. Aşkın anlamını biliyorum. 96. Gerekirse risk alacağım. 97. Sınavla ilgili bilgi vermeliyiz. 98. Misafirler otelin hizmetlerinden faydalanabilirler. 99. Çocuklarla konuştuğumda umutla doluyorum. 100. Yarama bandaj yapıştıracağım. International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 393 Appendix B Questionnaire about using the BNC in ESL Translation A1. Background information Name: Surname: Age: Gender: Male______ Female______ B. Reactions to using the BNC The following questions are regarding your opinions on using the BNC. Please use the scale below to circle the response that most closely resembles your perspectives. 1: strongly disagree 2: disagree 3: somewhat disagree 4: somewhat agree 5: agree 6: strongly agree 1 If I had known the corpus earlier, my translation skills would have been better 1 2 3 4 5 6 2 The real texts in the corpus were too difficult to understand. 1 2 3 4 5 6 3 I had some difficulty in analyzing concordance output. 1 2 3 4 5 6 4 I had some difficulty in analyzing collocate output. 1 2 3 4 5 6 5 I had some difficulty in using the corpus due to too many sentences in concordance output 1 2 3 4 5 6 6 I had some difficulty in using the corpus due to time and effort spent on analyzing the data 1 2 3 4 5 6 Akkoyunlu & Kilimci 394 7 Overall, the corpus is a very useful resource for my translation training. 1 2 3 4 5 6 8 Using the corpus is helpful for learning to translate. 1 2 3 4 5 6 9 I had some difficulty in using the corpus due to cut-off sentences in concordance output 1 2 3 4 5 6 10 I recommend using the corpus in the same course in the future. 1 2 3 4 5 6 11 Paper-based handouts of selected concordance lines were useful for me to capture the additional meanings of collocations 1 2 3 4 5 6 12 The corpus is more helpful than a dictionary for my translator training. 1 2 3 4 5 6 13 When I am unsure about the usage of a collocation, I will search for it in the corpus. 1 2 3 4 5 6 14 Corpus use should be taught in English classes. 1 2 3 4 5 6 15 Using the corpus is helpful for learning the meaning of collocations 1 2 3 4 5 6 16 I could have easily found the collocations in the concordance output without the instructor's help 1 2 3 4 5 6 17 The searching technique was easy to learn. 1 2 3 4 5 6 18 The practice sessions were helpful for learning the technique. 1 2 3 4 5 6 19 When I am unsure about the meaning of a collocation, I will search for it in the corpus. 1 2 3 4 5 6 20 I will use the corpus for my translation practices in the future. 1 2 3 4 5 6 International Online Journal of Education and Teaching (IOJET) 2017, 4(4), 369-396. 395 21 I will recommend the corpus to other students in my department or elsewhere. 1 2 3 4 5 6 22 Paper-based handouts of selected concordance lines are necessary to save time. 1 2 3 4 5 6 23 I will often use the corpus by my own choice. 1 2 3 4 5 6 24 I could have easily found out the meanings of the collocations in the concordance output without the instructor's help. 1 2 3 4 5 6 25 The corpus should be introduced in all ESL translation courses. 1 2 3 4 5 6 26 I had some difficulty in using the corpus due to the limited number of sentences in the concordance output 1 2 3 4 5 6 27 As I have learned more about the corpus, I have come to like it more. 1 2 3 4 5 6 28 The instructor’s guidance was useful for me to find the different meanings of the collocations in concordance output. 1 2 3 4 5 6 29 I had some difficulty in using the corpus due to unfamiliar vocabulary on concordance/collocate output. 1 2 3 4 5 6 30 I understand the purpose of using the corpus in this course. 1 2 3 4 5 6 31 Using the corpus is helpful for learning the usage of collocations. 1 2 3 4 5 6 32 I want to use the corpus in my other ESL courses too. 1 2 3 4 5 6 Akkoyunlu & Kilimci 396 Appendix C Indirect Exposure Activities A sample of corpus-derived worksheets on “Have” + noun collocations worksheet Complete the sentences using the words in the boxes. More than one answer can be possible; try to choose the most suitable option for each gap. 1. In one way or another, all these therapies seem to have an___________ on the electrical balances of the body. 2. Now that buses are no longer designed to enhance the streets they serve, Manchester has a ____________ to make a fresh start with its trams. 3. I felt lost for words; had ______________ breathing. 4. He had no ____________ what he would do after that. 5. It has the _____________ of being close to most of London's tourist attractions. 6. She said that it was increasingly important that people had easy ___________to information. 7. My workforce has a ______________ for being committed to the company. 8. The British Geological Survey has ____________ of working in every part of Britain. 9. ‘I want to make love to you more than anything in the world, but you have ____________ about me and I want everything crystal-clear between us, so ask your questions, Gemini girl.’ 10. We all have ______________ and we all have aches and pains. problems reputation difficulty chance effect doubts advantage experience idea access