Iberica 13 Ibérica 21 (2011): 71-92 ISSN 1139-7241 Abstract Teaching second language learners how to write research reports constitutes a crucial component in programmes on English for Specific Purposes (ESP) in institutions of higher learning. One of the rhetorical segments in research reports that merit attention has to do with the descriptions and justifications of sampling procedures. This genre-based study looks into sampling delineations in the Method-related sections of research articles on the teaching of English as a second language (TESL) written by expert writers and published in eight reputed international refereed journals. Using Swales’s (1990 & 2004) framework, I conducted a quantitative analysis of the rhetorical steps and a qualitative investigation into the language resources employed in delineating sampling procedures. This investigation has considerable relevance to ESP students and instructors as it has yielded pertinent findings on how samples can be appropriately described to meet the expectations of dissertation examiners, reviewers, and supervisors. The findings of this study have furnished insights into how supervisors and instructors can possibly teach novice writers ways of using specific linguistic mechanisms to lucidly describe and convincingly justify the sampling procedures in the Method sections of experimental research reports. Keywords: genre analysis, applied discourse analysis, research reports, writing instruction, academic writing. Delineating sampling procedures: Pedagogical significance of analysing sampling descriptions and their justifications in TESL experimental research reports Jason Miin-Hwa Lim Malaysian University of Sabah (Malaysia) drjasonlim@gmail.com 71 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 71 Ibérica 21 (2011): 71-92 JaSON MIIN-HWa LIM Resumen Concretando los procedimientos de muestreo: importancia pedagógica del análisis de las descripciones de muestreo y sus justificaciones en los informes de investigación experimental en la enseñanza del inglés como segunda lengua Enseñar a los estudiantes de una segunda lengua cómo escribir informes constituye un componente crucial en los programas de Inglés para Fines Específicos (IFE) que se imparten en instituciones de educación superior. Uno de los aspectos retóricos en los informes de investigación a los que se debe prestar atención guarda relación con las descripciones y justificaciones de los procedimientos de muestreo. Este trabajo, basado en estudios de género, examina los pasos retóricos referentes al muestreo que figuran en la sección de métodos de los artículos de investigación relacionados con la enseñanza del inglés como segunda lengua, escritos por investigadores expertos y publicados en ocho prestigiosas revistas académicas internacionales. Usando como marco de referencia los trabajos de Swales (1990 y 2004), hemos llevado a cabo, por un lado, un análisis cuantitativo de los pasos retóricos y, por otro, una investigación cualitativa de los recursos del lenguaje que se emplean para concretar y definir procedimientos de muestreo. Esta investigación tiene considerable relevancia para los estudiantes y académicos de IFE al haberse obtenido conclusiones pertinentes relativas al modo más adecuado en el que se pueden describir las muestras objeto de estudio, satisfaciendo de este modo las expectativas de los examinadores, evaluadores y directores del trabajo de investigación elaborado. Los hallazgos de este estudio aportan claves que ayudan a los directores y profesores en su labor docente para con los escritores noveles; concretamente, cómo utilizar mecanismos lingüísticos específicos para describir con claridad y justificar de forma convincente los procedimientos de muestreo en las secciones de los informes de investigación experimental que se ocupan de la descripción del método de investigación Palabras clave: análisis de género, análisis de discurso aplicado, informes de investigación, docencia de la producción escrita, escritura académica. Introduction Writing dissertations often forms a crucial part of undergraduate and Master’s programmes in the teaching of English as a Second Language (TESL) and other fields related to language education. Novice writers, however, frequently encounter problems while presenting new information in the early chapters of a dissertation, particularly the introductory and 72 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 72 methodological chapters. This study was therefore motivated by my concerns about the language difficulties faced by second language dissertation writers in writing the drafts of dissertations submitted to supervisors for comments and corrections. One of the information elements containing numerous language errors has to do with the delineations of sampling procedures in the Method section of a research report. In recent genre-analyses (e.g., Flowerdew, 2005; Kanoksilapatham, 2005; Samraj, 2005; Lim, 2006; Ding, 2007), the word “delineation” or “delineating” has been used to carry the meaning of “describing or reporting something (particularly a procedure, move, or rhetorical structure) in great detail”. as errors are often attributed to learners’ understanding of the circumstances under which samples were collected and their command of the language, some instances of errors (committed by undergraduate learners) need to be cited here to demonstrate the need to study experienced writers’ language choices associated with this communicative move. The following examples illustrate some authentic errors committed by undergraduate novice writers in the delineations of sampling procedures in their final year dissertations. While these errors have been highlighted with italics, their associated corrections, replacements, additions, and/or explanations are indicated in parentheses as shown below: (1) The researcher decided to use what is (to be deleted) random sampling. This technique is (was) more useful to select (for selecting) the respondents from all the population (Nardi, 2003). The researcher selects (selected) all the Form Four students because there were only three classes of Form Four in that school. (2) The Form Four ESL learners was (were) chosen because they are (were) capable to read (capable of reading) the questionnaire in (the) English language, and if they do (did) not understand (the text given) at least it is (it would be at least) easier to conduct (guide) them. The aforementioned instances have illustrated that language difficulties are not merely restricted to tense usage, but may include mistakes in the use of vocabulary items, lexical chunks, and phrasal combinations. While it has to be acknowledged that numerous recent studies have focused on analysing errors in learners’ language usage and difficulties (Döpke, 1999; Ellis, 2006; collins, 2007; Lim, 2007), linguists and genre analysts such as Bhatia (1993), Berkenkotter and Huckin (1995), Hudson (2007) or Swales (1990 & 2004) emphasise the significance of acquiring “situated knowledge” and “genre DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 73 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 73 knowledge” while learning language in various academic contexts. This can be done through comprehending sufficient examples of authentic text segments (used by expert writers) in close relation to the communicative functions of the rhetorical segments concerned. The aforementioned segments appear in the form of rhetorical moves, each of which comprises several possible steps signifying the writers’ communicative functions that are relevant to the specialised discourse community. a “move” here is defined as “a rhetorical unit that performs a coherent communicative function in a written or spoken discourse” (Swales, 2004: 228). although a move may be realised in the form of a clause, a sentence or several sentences, Swales (2004) has pointed out that it is not a formal unit but a functional one. Several rhetorical steps (under a move) may then perform different specific functions, all of which accomplish the same principal function of the move, which constitutes a hierarchically higher functional unit. In the context of this study, analysing the aforementioned segments associated with sampling delineations may provide us with adequate related instances that can be used as (i) examples in pre-writing instructional sessions, and (ii) frames of reference in post-writing corrections and explanations. To comprehend the status of these rhetorical segments, I will first review some genre-based studies connected with sampling delineations. In the Method section, “delineating/describing the sample” was given the status of a “step” within a move rather than a move by itself in some disciplines such as medicine (Nwogu, 1997) and management (Lim, 2006). Nevertheless, it would be interesting to find out whether this rhetorical category is so inextricably linked with other related steps that it can always be aptly viewed as parts of a particular move (i.e., “describing data collection procedures”). Motivated by such concerns, this study aims to (i) determine whether “delineating the sample” is a stable move occurring in most of the journal articles on TESL, (ii) ascertain the possible ways in which these sampling procedures are justified, and (iii) identify the salient linguistic mechanisms that experienced writers frequently use to describe and justify these sampling procedures. In relation to the aforementioned objectives, some past research needs to be reviewed to indicate (i) the general prevalence of “delineating data collection procedures” in several disciplines, and (ii) the prominent characteristics of these steps. While Holmes (1997) found that the Method sections are rare (2 JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9274 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 74 out of 10) in history research articles (ras), Posteguillo (1999) reported that computer science articles in his corpus do not include the Method section at all. In contrast, the biochemistry Method sections studied by Kanoksilapatham (2005) contained a 4-move structure consisting of “describing materials”, “describing experimental procedures”, “detailing equipment”, and “describing statistical procedures”. In her study, the segments referring to the sample were given the functional label “describing materials” (Kanoksilapatham, 2005: 277) rather than “describing the sample” (Lim, 2006: 287). Nonetheless, the term “materials” in the context of biochemistry articles also refers to a “sample” (of natural substances, human/animal organs or tissues, or chemicals whose source and/or background are often described) analysed instead of merely a set of objects needed in collecting or analysing data. The extent to which the significance of sampling descriptions varies across disciplines can be further considered via a comparison of the prevalence of sampling descriptions in biochemistry Method sections with that in (i) medical research methods studied by Nwogu (1997), and (ii) management Method sections analysed by Lim (2006). Nwogu (1997), in particular, provided a three-move structure for the Method sections of medical research papers, in which the major communicative moves were (i) “describing data-collection procedures” (Nwogu, 1997: 128) involving medical researchers’ specifications of the source of data, sample size, and criteria for data collection); (ii) “describing experimental procedures” (Nwogu, 1997: 129) involving logical and sequential descriptions of steps and procedures during the experimental process; and (iii) “describing data analysis procedures” (Nwogu, 1997: 130) in which tools used in statistical or quantitative studies were identified and accounted for in relation to the body of the research data. The emerging problem is that “describing experimental procedures” in Move 2 (specified by Nwogu (1997) as a move that occurred mainly in experimental studies) is a functional label that also encompassed the meaning of “collecting data via experiments” in Move 1. This means that Move 2 in Nwogu’s study (1997) can be reckoned to be a step of Move 1 given that descriptions of experimental procedures in stages actually form part of the on-site “data collection procedures” (for experimental studies). It would therefore be interesting to use a corpus in a discipline, like TESL, to investigate whether provision of details concerning the source of data and sample characteristics might actually constitute a separate or distinct move DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 75 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 75 that does not involve any on-site data collection procedure (in the experimental processes itself). In relation to this, Lim (2006) reported that “describing the sample” is one of the three steps found in the initial move called “describing data collection procedures” – the other two steps being (i) “recounting steps in data collection” and (ii) “justifying data collection procedures”. Despite his illustration using instances of the rhetorical categories in management research articles, two problems remain in regard to (i) whether descriptions of sampling procedures (also regarded by Lim (2006) as part of data collection procedures) actually constitute part of the data gathering procedures in other disciplines (apart from management) in terms of communicative functions and sectional organisations, and (ii) whether the linguistic features of “describing the sample” are so distinctly different from those of “recounting data collection/gathering procedures” that these two rhetorical categories actually need to be considered as separate moves (rather than steps within the same move). More importantly, as genres are dynamic and open to change in response to users’ needs and changes in the contexts in which they occur (Berkenkotter & Huckin, 1995; Paltridge, 2000), it would be interesting to investigate how a particular rhetorical category is expressed in a certain discipline. as genres become recognizable only after they have become “somewhat standardised” (Bhatia, 1995: 1), experimental articles related to TESL, which form an established and standardised subgenre of applied linguistics ras, may therefore provide useful information on (i) the extent to which sampling delineations constitute a separate move, (ii) its detailed communicative functions and the frequency of its related justifications, and (iii) the linguistic mechanisms used to accomplish the related communicative functions. Motivated by the need to enlighten second language learners in their reading and writing of the Method sections in TESL experimental studies, this genre-based study seeks to answer three research questions as follows: (1) What are the communicative functions of “delineating the sample” in experimental research papers on TESL? (2) Do the frequencies of sampling delineations in TESL experimental reports largely hinge on the types of headings that the writers use? JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9276 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 76 (3) What salient language mechanisms are used to delineate the sample in research papers on TESL? Research method To obtain data pertaining to the aforementioned research questions, a total of 32 TESL-related articles on experimental research were selected from eight different international refereed journals published from 2004 to 2008, including Applied Linguistics, TESOL Quarterly, Studies in Second Language Acquisition, Language Teaching Research, Journal of English for Academic Purposes, International Review of Applied Linguistics in Language Teaching, System, and RELC Journal. The purposive sample, comprising four articles from each journal, was selected using my “experience and knowledge of the group to be sampled” (gay, Mills & airasian, 2009: 134), and the selection criteria were that the articles had to be (i) those published in established international refereed journals, and (ii) closely connected with experimental research in the teaching of English. In some of these journals, the research procedures are presented in sections under investigation-focused headings, such as “The Study” or “The Experimental Study”, but in accordance with Lim (2006) and Pho (2009), they are considered as “Method sections”. The findings obtained on the delineation of sampling procedures are therefore generalisable only to TESL articles bearing the aforementioned characteristics. Two specialist informants who had published TESL experimental research articles in established international refereed journals were interviewed to (i) provide views on the rationale for describing sampling procedures in the discipline, and (ii) ascertain the degree of acceptability of the communicative functions involved in sampling descriptions. The informants’ spoken data in the face-to-face interviews were recorded digitally and studied to “triangulate the discourse analyses” (Berkenkotter, 2009: 13) of the TESL experimental research reports. The overall organisation of the articles was analysed before attention was focused on studying (i) the generic structure of the sections containing sampling delineations, and (ii) the linguistic exponents employed to realise each rhetorical move and constituent step. Swales’s (1990 & 2004) seminal “move- step analysis” was first used to examine the texts using a contextual procedure that emphasised communicative purposes recognised by expert members of an academic discourse community. Using the approach, I analysed the genre in DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 77 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 77 terms of distinct units in a hierarchically organised framework whereby a section was divided into rhetorical moves that were subsequently broken down into constituent steps. This study first attempted to distinguish “delineating the sample” from other co-occurring rhetorical categories in the Method sections. The minimal unit to which a main rhetorical function could be assigned was a T-unit, which is “one main clause plus any subordinate clause or nonclausal structure that is attached to, or embedded in it” (Hunt, 1970: 4). More generally, each T-unit analysed in this study was “an independent clause and all of its dependent clauses” (Sachs & Polio, 2007: 79). This means that each rhetorical step is “minimally” a T-unit consisting of one main clause in some cases, although it may comprise several sentences or paragraphs (with the same rhetorical function) in other cases. a different rhetorical step incorporated in a subordinate/dependent clause was considered as having been embedded in the step found in the main/super-ordinate clause(s) of the T-unit. Subsequently, occurrences of each step were marked in each text so that its frequency could be identified. Typographical features, division of sections and subsections, and linguistic features were used to distinguish this move from others (Mauranen, 1993; connor, Davis, & de rycker, 1995; Nwogu, 1997; connor & Mauranen, 1999). a step constituting a segment might consist of a main clause or even several sentences insofar as its occurrence was not interrupted by any other rhetorical step. attention was then focused on all segments associated with sampling descriptions and/or justifications (if any). Each segment pertaining to sampling delineations were analysed to ascertain whether other moves (particularly those associated with on-site gathering of data) were embedded in them. The number of occurrences of sampling delineations was counted with reference to the number of times a step appeared without being interrupted by any other step. Mann-Whitney U-tests were conducted to ascertain the extent to which the frequencies of the steps associated with sampling delineations differ in accordance with the major headings under which the related segments appear. Salient linguistic features were then analysed with reference to sentence structures, clause elements, categories of phrases, and parts of speech if they appeared as prominent features of the rhetorical category. The analysis of prominent linguistic choices was conducted on the basis of (i) linguistic descriptions provided by Quirk et al. (1985), and greenbaum and Quirk (1990), and (ii) descriptions of academic language as illustrated by Thomas and Hawes (1994) and Lim (2006, 2008 & 2009) for the research genre. JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9278 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 78 Results and discussion Based on the analysis, two major steps associated with the delineation of sampling procedures have been identified. While the first step (i.e., “describing the sample/participants”) pertains to sampling criteria and descriptions of the location, size, and proficiency-related characteristics of a sample, the second step (i.e., “justifying the sampling procedures”) focuses on the writer’s demonstrations of the comparability of treatment groups and highlights advantages of employing the sample. consulted on the aspects covered in research procedures, both specialist informants acknowledged the frequent inclusion of these two distinctly separate rhetorical steps on sampling delineations. The communicative functions found in the step analysis and endorsed by the specialist informants are illustrated in Table 1. Step no. Communicative functions Specialist Informant a (SIa) has pointed out writers’ tendency to consider the “overall design”, which she called the “conceptualization of the whole study”, before moving on to sampling descriptions and deciding on the material/s or items to be used in the instrument. This has supported my categorisation of sampling descriptions as a distinctly separate move from descriptions of overall research designs and descriptions of materials and/or instruments. Specialist Informant B (SIB) has commented that “sample is separated from procedures for instrument development, and for actually administering the test” and “so we have subjects, instruments and procedures”. In this context, she referred to “procedure” as the actual “administering of instrument” (not descriptions of the instrument) at the research site where data were collected. This has again substantiated my decision to categorise sampling delineations as being different from the moves associated with the descriptions of materials and instruments and DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 79 Step no. Communicative functions Step 1: Describing the sample/participants (a) Describing the location of the sample/participants; (b) Describing the size of the population; (c) Describing the characteristics of the sample (sizes, origins, age groups/levels, educational backgrounds, socio-economic levels, language proficiency levels, language used (frequency of usage), training and qualifications, experience, assessment criteria for grouping, requirements, etc.); (d) Describing the sampling criteria/techniques Step 2: Justifying the sampling procedure/s (a) Demonstrating comparability of the treatment groups; (b) Highlighting advantages of using the sample. Table 1. Communicative functions of “delineating the sample” in TESL experimental research articles. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 79 those of on-site data collection procedures. In regard to justifications, SIa has considered the incorporations of justifications as optional and dependent on the need of the writer. SIa has also specified that when a procedure “is not really mainstream” and is considered as not readily accepted by the research community, it would be necessary to justify the procedure concerned by citing the past researchers who adopted them although few details of past research methods may be incorporated. given the aspects illustrated above, we can now study the overall distribution and frequencies of the steps to provide an overview of the degrees of prevalence of sampling delineations in Method sections 1 through 32 (i.e., M1 – M32). Table 2 shows that “describing the sample/participants” appears in most (i.e., 31 out of 32 Method sections) of the TESL experimental research articles, with 2.09 occurrences per section (i.e., 67 occurrences in a corpus of 32 Method sections). “Justifying the sampling procedures”, however, occurs in nearly half (i.e., 14 out of 32) of the research reports, with 0.53 occurrence per section. Mann-Whitney U-tests were conducted to identify the inter-heading differences in the occurrences of both steps in the entire corpus. Mann- Whitney U-tests were used instead of independent samples t-tests (although the number of occurrences is a ratio variable) because the occurrences were not normally distributed for all the constituent steps. Table 3 shows the Mann-Whitney U-statistics and asymptotic values for each of these steps. as the asymptotic values for all the steps in the Method sections are above the cut-off point of 0.05, I have found no significant inter-heading differences in the occurrences of sampling delineations. Occurrences of both steps therefore exhibit no significant differences across papers with procedure-focused headings (e.g., “Method/s”, “Methodology”, “research Design”, etc.) or investigation-focused headings (e.g., “The Study”, “The Experiment”, “The experimental Study”, etc.). Having justified the inclusion of the segments related to research methods, the following sections focus on qualitative results for each of these steps. Article no. Heading for the Results section Step 1 Step 2 Steps 1-2 JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9280 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 80 Step 1: Describing the sample/participants Step 1 often (i) precedes segments pertaining to descriptions of research instruments and other on-site data collection procedures, and (ii) includes descriptions of the characteristics of the participants, and the locations DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 81 Article no. Heading for the Results section Step 1 Step 2 Steps 1-2 M1 Method (Three Experiments) 3 1 4 M2 Methods 2 0 2 M3 Method 2 0 2 M4 Experiment 1 & Experiment 2 2 1 3 M5 Method 3 2 5 M6 Method 2 0 2 M7 Experiment 1 & Experiment 2 3 0 3 M8 Method 2 2 4 M9 Methodology 6 2 8 M10 Study 1 & Study 2 2 1 3 M11 The Current Study (2 Analyses) 4 0 4 M12 Method 2 0 2 M13 Method 1 1 2 M14 The Experiment 2 0 2 M15 Method 2 1 3 M16 The Study 3 0 3 M17 The Experimental Study 3 0 3 M18 The Experimental Study 1 1 2 M19 Design of the Study 3 1 4 M20 Method 2 0 2 M21 Experiment I & Experiment 2 0 0 0 M22 Experiment 1 & Experiment 2 1 1 2 M23 Experimental Design 1 0 1 M24 Method 1 0 1 M25 Research Method 2 0 2 M26 The Experiments 2 1 3 M27 Research Design 1 1 2 M28 Method 1 1 2 M29 Method 2 0 2 M30 Methodology 1 0 1 M31 An Experiment 3 0 3 M32 The Study 2 0 2 No. of TESL papers containing the step/s 31 14 31 Frequency of step/s under procedure-focused headings 39 12 51 Frequency of step/s under investigation-focused headings 28 5 33 Total frequency of step/s in all Method sections 67 17 84 Mean frequency 2.09 0.53 2.63 Median frequency 2.00 0.00 2.00 SD 1.118 0.671 1.431 Table 2. Frequencies of “describing the sample/participants” and “justifying the sampling procedures” in TESL experimental research papers. Rhetorical Category Mann-Whitney U Statistic (U) Asymptotic Value (2-tailed) Move 2-Step 1 104.500 0.440 Move 2-Step 2 105.000 0.422 Move 2 105.000 0.454 Table 3. Asymptotic significance values indicating inter-heading differences with reference to occurrences of the procedural steps. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 81 where the samples were obtained. Sampling descriptions rarely appear under the section heading “Procedure” or “Data collection Procedure” (except in Method sections 11, 25, and 31) as they generally constitute a separate subsection under a separate heading such as “Participants” or “Subjects” in half of the TESL experimental papers. In all cases, sampling delineations are always separated from on-site data gathering procedures, and they usually precede descriptions of data collection procedures in most (i.e., 19 out of 32) of the Method sections. In some Method sections, although brief on-site data collection procedures are described in sections under such headings as “Participants” or “Subjects”, they only appear after descriptions of the participants have been presented. Sampling descriptions in TESL experimental research methods constitute a separate communicative move in that such descriptions do not incorporate descriptions of a series of stages in which the data were collected on the research site, or the preparation of the instruments used in collecting data. analysing the grouping methods alone reveals a key characteristic of “describing the sample” which is frequently found in the Method sections of other disciplines such as management (e.g. Lim, 2006). There are, however, characteristics typically found in the Method sections of TESL experimental ras but not in those of other disciplines (see Table 4). JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9282 Subject (noun phrase denoting participants) Predicator (copular verb in the past simple) Complement (NP describing participants and containing a headnoun premodified or postmodified by adjectival prepositional or participial phrases) Adverbial/s (prepositional phrase/s indicating origin/s or research site/s or participial phrases indicating research sites or residential areas) The participants in the second trial were 62 Dutch-speaking students (aged 19–20) of modern languages (majoring in English) at the same college for translation and interpreting as in the first experiment… (M1: 251) The participants in this study were 121 students learning English as a foreign language in Fukuoka, Japan… (M3: 51) Individuals in this study were 154 ESL students attending intensive programs at four U.S. universities. (R6: 278) Participants were 32 students of modern languages, majoring in English, at a college for translation and interpreting in Brussels, Belgium. (M15: 248) The participants were 49 students enrolled in general English classes in a national university in Japan. (M28: 357) All were Cantonese, L1 speakers, of Chinese ethnic background, living in high-rise housing blocks in medium income areas of the New Territories of Hong Kong. (M31: 332) Table 4. Subject-predicator-complement-adverbial (SPCA) structures used in descriptions of participants in Step 1. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 82 For instance, the descriptions of the participants’ proficiency levels in the SPca structure constitute a distinguishing feature of this step in TESL experimental methods. These typical descriptions of participants generally include a predicator consisting of a plural copular verb in the past simple, preceded by a sentence-subject comprising a shorter noun phrase denoting participants. The copular predicator is generally ensued by a more elaborate noun phrase describing participants, in which the headnoun is postmodified by participial phrases (e.g., “enrolled in general English classes”, “attending intensive programs”, etc.) and adjectival prepositional phrases (e.g., “of chinese background”, etc.) closely associated with language-related descriptions. The headnoun is often premodified by numeric figures and language-related adjectives (e.g., ESL, cantonese, etc.). The sentence-final spatial adverbial is usually a prepositional phrase indicating the research site where the participants were involved. aside from the copular verbs, the use of transitive lexical verbs in the past tense also constitutes a salient feature of Step 1. Such instances are shown as follows: (1) all the learners reported previous English instruction … (M2: 50) (2) all participants reported limited exposure to English outside of school, a situation consistent with foreign language learning. (M9: 551) (3) These participants (mean age: 41.7; range: 18.1–61.0) reported using English on average only about 5% daily (0.23%), estimating their daily use of French at a mean of 95% (80–100%) ... (M11: 424) (4) However, a smaller proportion of the migrant students (53%) claimed to have had formal instruction at the private language schools ... (M16: 416) (5) Following graduation from the Military academy, all participants undertook the same post-graduate level of training in the army aviation School and Training center ... (M20: 42) These predicator-object phrasal combinations generally contain transitive lexical verbs (e.g., “reported”, “undertook”, etc.) ensued by sentence-objects indicating language exposure (e.g., “limited exposure to English”, “formal instruction at the private language schools”, etc.). It can be seen that descriptions of the participants’ language backgrounds may be categorised into three major aspects, encompassing descriptions prior to, during, and after the research period as exemplified in Table 5: DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 83 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 83 The above categorisation of language proficiency descriptions has shown that authors generally describe participants’ backgrounds in TESL experimental research by referring to the subjects’ language-related activities with four major temporal features. While the past simple is used for describing participants’ proficiency levels during the research period, the past progressive indicates activities continuing during the research period. The present simple or present perfect is used occasionally for language-related activities that have relevance at the time of research reporting whereas the past simple and past perfect may be used to delineate activities before the research period in order to provide comprehensive background information on the subjects’ proficiency in the target language. The characteristics of the participants are often described with reference to the origins or backgrounds of the subjects as shown below: (6) The learners came from various L1 backgrounds, including asian, romance, and germanic. Thus, the class makeup was typical of many university-level intensive English programs in the U.S. (M2: 50) JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9284 Syntactic choice Instance of references to hypotheses • The 28 ESL learners who took part in the study were enrolled in a university-level intensive English program. (M2: 410) • Students’ proficiency levels ranged from lower intermediate (50) to upper intermediate (51) to advanced level (53). Fourteen different native languages were represented, with the largest being Spanish (58), Arabic (22), Japanese (22), and Korean (11). (M6: 278) Using the past simple for descriptions of proficiency levels during the research period • Their proficiency in English was estimated to be of upper-intermediate to advanced level … (M15: 248) • In the second school, the participants were 30 in total. The subjects were studying English in a secondary Greek school (all at the age of 12-13) ... The participants selected in both schools were studying English ... (M13: 75) Using the past progressive for activities continuing during the research period • Except for three students from one of the private language schools, all students in this study were studying full-time … (M16: 418) • Students enrolled in these classes have high school or equivalent degrees from their home countries, began studying English as teenagers, and have been in the United States for between 0 and 6 months ... (M7: 327) Using the present simple/perfect for language activities with present relevance at the time of research reporting • University students in the Netherlands have typically had 6–7 years of English in high school; all participants could therefore be expected to have at least an intermediate level of proficiency in English. (M12: 460) • Fifteen students (11 Korean, 3 Japanese, and 1 Indonesian) enrolled in a high-intermediate English as a second language (ESL) class in the intensive English program (IEP) at a large Midwestern university participated in this study … Participants had been in the United States for a range of 1 month to 1 year, and most of them had recently been placed in the high-intermediate level based on the IEP placement test … (M10: 75) Using either the past simple or the past perfect for language- related activities before the research period • The learners in the fourth group, the “Early” group, were child (early) learners of English. They had arrived in the U.S. as children, at about 9 years of age, had resided in the U.S. for a mean of 11 years, … (M22: 235) Table 5. Syntactic choices used for describing language proficiency in Step 1. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 84 (7) The participants were selected from four second-year EFL classes at Kyushu University … (M3: 51) (8) The students were drawn from both international and immigrant ESL populations …(M8: 261) (9) The participants were individually recruited from students (age range 18 to 20) at a technical college in Japan …(M24: 15) (10) The subjects were all from “Band 1” English medium secondary (High) schools, in an education system … (M31: 332) The expressions illustrated in (6)-(10) show that phrasal combinations comprising past tense verbs and prepositions indicating source or origin (e.g., “were drawn from”, “were recruited from”, etc.) constitute an important feature of this step in which more specific delineations of the participants, especially those concerning nationalities, classes, program areas, and types of college are subsequently presented. an alternative structure used to highlight participants’ involvement is the use of procedural verbs denoting participation as illustrated in Table 6. In the cases where the past tense procedural verb is used, meticulous descriptions of the participants appear in the pre-predicator rather than the post- predicator position. DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 85 Subject (noun phrase denoting participants with pre-modifiers and post-modifiers) Predicator (copular verb in the past simple) Adverbial/s (prepositional phrase/s or infinitive clauses) Forty-four students who had not participated in the rating exercise participated in the main experiment. (M4: 217) Eighty native Dutch-speaking students (51 female, 29 male) aged 18–29 (M = 21.5, SD = 2.46) at the University of Amsterdam participated in the rating session, which was conducted as part of a 1-h test session at the Department of Psychology. Participants received !7 (approximately US$8) for their participation. (M12: 460) A total of 104 students, who were enrolled in an introductory psychology course at the University of Massachusetts, participated to fulfil a requirement for that course … (M14: 136) Three intact classes of 99 first- year Japanese public high-school students (53 males and 46 females) aged 15 and 16 participated in this study. (M30: 110) Table 6. Subject-predicator-adverbial (SPA) structures with detailed descriptions of participants appearing in the sentence-initial subjects in Step 1. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 85 The sentence-initial subjects generally appear heavily loaded with numerical and education-related pre-modifiers (e.g., “99 first-year Japanese public high- school”) and other adverbial post-modifiers which are either prepositional phrases (e.g., “at the University of amsterdam”) or relative clauses (e.g., “who had not participated …”). In sampling descriptions, group formation usually becomes an indispensable feature, but a wide range of syntactic choices can be employed to accomplish the assignment of subjects into groups. Descriptions of grouping methods in Step 1 can be presented using (i) a passive verb denoting usage ensued by a preposition of purpose, (ii) passive phrasal verb denoting assignment, (iii) a passive verb phrase denoting choice, or (iv) a passive phrasal verb indicating division in the sentence-predicator position. Instances illustrating group selection and assignment are given in Table 7. Of these syntactic choices, the use of passive phrasal verbs denoting assignment (e.g., “were assigned to”), verb phrase denoting choice (e.g., “were selected”) and phrasal verb indicating division (i.e., “were split into”, etc.) appear more prominent. JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9286 Syntactic choice Instance of segments indicating methods of grouping • Two parallel groups with a shared history of English courses at the college were used as experimental (N=27) and control (N=35) groups. (M1: 248) Employing a passive verb denoting usage ensued by a role- related preposition of purpose • Two parallel groups sharing the same English courses were used as experimental (N=39) and control (N=23) groups. (M1: 251) • Participants belonged to three groups: 10 L1 Arabic speakers (Arabic), 10 non-Arabic ESL learners (non-Arabic ESL), and 10 native speakers of English (English)…(M7: 327) Listing groups after using a verb denoting relationship or formation • Out of six intact classrooms, three groups were formed: the direct-only correction group (n = 31), the direct metalinguistic group (n = 32), and the control group (n = 28). (M8: 261) • One of the two classes was randomly assigned as a control group, and the other as an experimental group. (R2: 411) Using a passive phrasal verb denoting assignment • These teachers were assigned to two experimental groups (recasts vs prompts) and a control group… (M9: 551) • These were selected after conducting preliminary observations in six intensive ESL classes in six schools. (M9: 550) Using a passive verb phrase denoting choice as the sentence-predicator • A total of eight students were randomly selected to participate in the oral proficiency assessment tasks…(M29: 347) • The learners were randomly distributed into two intact speaking and listening elective classes by the language program administrators ... (M2: 410) • They were split into three groups using a random procedure from the beginning of the experiment: PI group (n =15); TI group (n= 15); MOI group (n = 17) …(M13: 75) Employing a passive phrasal verb indicating division in the sentence- predicator position • All students were from the same class, that is, form three students from a Chinese medium, low banding secondary school, and were divided into two groups of four … (M29: 347) Table 7. Syntactic choices used in indicating methods of grouping subjects in Step 1. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 86 Like the use of the phrasal combinations above, subjects’ backgrounds in TESL experimental research are often depicted using the past perfect as illustrated in (11)-(13): (11) all of the participants had studied English for a minimum of seven years, and had scored 80 per cent or higher on the 2,000 word level of Version 1 of the Vocabulary Levels Test (Schmitt 2000). Their mean score was 27.4 indicating that they had mastered that level ... (M3: 51) (12) all were NSs of Quebec French and were born and raised in Montreal (n = 6) or in granby, Quebec (n = 34), in homes where only French was used. all participants had received primary and secondary education in French ... (M11: 413) (13) all the students had completed 6 years of English study, involving approximately 800 h of classroom instruction, before entering university... (M28: 357) This is a more distinct characteristic that illustrates the extent to which authors place greater emphasis on participants’ previous experience using the past perfect in the description of the sample. Such references to past training of the participants provide readers with pertinent information on the educational backgrounds of the subjects, particularly in relation to language proficiency. Step 2: Justifying the sampling procedure This step differs from Step 1 in that it moves beyond the mere description of a research activity, and hence, it is appropriate to consider it as an explanation intended to justify sampling techniques (see Figure 1). DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 87 Describing the sample/participants Justifying the sampling technique One of the two already established groups (17 students) was randomly assigned to the experimental condition, while the other group (15 students) was assigned to the control condition. To ascertain that both groups were initially on a par as far as oral proficiency was concerned, we compared their scores in the oral proficiency interviews that were part of the exam session at the end of the first year of their training (three months prior to the start of the experiment). (M15: 249) Out of a population of 85 students, we selected four sub-groups of 10 students based on the results of an in-house ESL placement test. (M27: 88) We restricted participants to those with intermediate level scores in order to obtain as homogeneous a sample as possible. (M27: 88) Figure 1. Shifts from sampling descriptions in Step 1 to sampling justifications in Step 2. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 87 This is a step that works in close relation with Step 1 to stifle possible doubts or scepticism about the acceptability of the selection of the participants involved in TESL experimental research. Justifications of sampling methods often centre on how the comparability of the groups of participants was ensured (before further analysis could be carried out) via careful pre- instructional comparison or restriction (control) of participants with reference to their proficiency levels. There is no denying that in some cases (as shown in Table 8), justifications may be more implicit as comparability of the treatment groups are indicated without using overt linguistic choices, yet in most cases prominent lexical and syntactic choices normally characterise this step. While justifying the selection of a sample, authors may attempt to justify the selection using (i) purpose adjuncts (e.g., “in order to eliminate one of the methodological questions …”) or (ii) reason adjuncts (e.g., “because they represented a block of learners whose familiarity with English could be regarded as limited”) after verb phrases denoting choice (e.g., “were chosen”) JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9288 Linguistic choice Instance of segments used in sampling justifications • To ascertain that both groups were initially on a par as far as oral proficiency was concerned, we compared their scores in the oral proficiency interviews … (M15: 249) • This group differed from the Late+10 group only in their age of arrival (and chronological age) and was included in this study to examine the influence of AOA on the perception-production relationship. (M22: 235) • We restricted participants to those with intermediate level scores in order to obtain as homogeneous a sample as possible. (M27: 88) Using purpose adjuncts in the form of infinitive clauses • To ensure this, the following steps were taken…only words that I thought were worth learning were used so as to increase the pedagogical value of the study for the participants … (M6: 280) • These eight students were chosen based on their level of proficiency, their L1, and their scores in this study. Of the eight students, three were in the top third of posttest scores, three were in the middle third, and two were in the lower third ... (M6: 284) Using a verb phrase denoting choice and a reason adjunct • They were chosen because they represented a block of learners whose familiarity with English could be regarded as limited; all had similar scores of between 25 and 36 out of 80 on the school’s entry test. (M26: 370) • One reason for our assumption is that in the 1990s and through the time our study was conducted, the United States was the most popular destination for Japanese high school and university students (“Consultants’ Report: Japan,” 2003) … In 2002, Riney read approximately 260 application letters for this program, in which students were asked to state their motives for applying ... (M5: 448) Using integral and non-integral citations of past researchers’ statements to support a sampling technique • We believe, however, based on our experience, that GA is one of the most widely heard exonormative varieties of English in Japan. Related literature supports this notion: McArthur (1996), regarding universities, television, and the media; Crystal (1997), regarding Hollywood and film ...(M5: 449) Table 8. Linguistic choices used for justifying sampling techniques in Step 2. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 88 in order to demonstrate the authors’ cautiousness in conducting various procedures that ensure the comparability of the groups involved in experiments. alternatively, writers may justify the sampling procedure by using integral or non-integral citations of past researchers’ statements concerning the aptness of the research site from which the participants were selected or the samples of data were obtained. Conclusions and implications for reading and writing instruction This study has resolved several major difficulties associated with the descriptions of sampling procedures in the writing of TESL research methods. “Delineating sampling procedures” has been found to be a separate and yet principal rhetorical move which is not embedded in descriptions of instruments and on-site data gathering procedures. Having resolved some inconsistency associated with the use of the functional label (i.e., “describing data collection”) for sampling descriptions in past studies (e.g., Lim, 2006; Nwogu, 1997; Pho, 2009), this study has demonstrated the need to view delineations of sampling procedures as a separate distinct communicative move that is different from descriptions of on-site data gathering procedures. This move is realised in two distinct ways via (i) descriptions of the sampling procedures which occur in almost all of the experimental papers, and (ii) sampling justifications that appear in nearly half of the research papers. Overall, “delineating the sample” occurs more than twice on average, thus indicating that it is a principal communicative category in the experimental studies. While Step 1 focuses on descriptions of the sample location, size, characteristics and sampling techniques, Step 2 demonstrates the comparability of treatment groups and highlights significant advantages of using the sample. The use of Mann-Whitney U-test has confirmed that the distribution of the occurrences of each step is not dependent upon the differences between the procedure-focused and investigation-focused headings of the Method- related sections. In experimental research, instructors may allow novice writers to incorporate sampling delineations under either procedure-focused headings or investigation-focused headings given that they do not entail significantly different frequencies of sampling delineations. In regard to specific semantic functions, Step 1 generally focuses on the descriptions of participants in a research site to enlighten readers on the DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 89 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 89 circumstances under which the study was conducted without incorporating any description of the steps taken in on-site data gathering activities. These descriptions of the characteristics of the participants are often linked to the participants’ perceptual judgments, learning contexts or language speaking environment to demonstrate their desired effects on research methods. This is often done via shifts from the past simple to the past perfect while revealing the previous experience of the participants. as the past perfect is frequently used in conjunction with the past simple by authors to underscore participants’ previous experience and past training, it is recommended that ESP instructors retain the contextual information on participants’ language proficiency levels in gap-filling items, so that learners can contrast the past perfect with the past simple in comprehensible and meaningful sampling descriptions in their discipline. In the tasks concerned, the range of syntactic choices to be incorporated in describing samples may comprise those differentiating the use of (i) the past simple for descriptions of proficiency levels during the research period, (ii) the past perfect for actions when writers look back on earlier actions, (iii) the past progressive for activities that began before the research period, and (iv) the present simple or present perfect for language activities with current relevance. In guiding learners to describe samples in Step 1, it would be interesting for ESP instructors to highlight how noun phrases denoting participants (in an SPca structure) are post-modified by participial phrases, adjectival prepositional phrases, and relative clauses. gap-filling exercises requiring learners to complete sentences involving the predicator-object combinations may also be appropriate tasks for training learners to describe the sample or participants’ language backgrounds. Novice learners may also need some training in using phrasal combinations encompassing prepositions indicating source or origin that help to delineate participants’ nationalities and educational backgrounds, which distinguish the Method-related sections of TESL experimental ras from those in other disciplines. as delineations concerning group formation often form a central component of experimental research, salient linguistic mechanisms to be introduced in classroom exercises may have to include the syntactic choices pertaining to group selection and assignment. Vocabulary items that may be given the focus in sentence construction exercises can also cover phrasal verbs denoting assignment and division, and role-related prepositions of purpose. In regard to Step 2, novice writers may learn to justify their samples via background reading of the papers in this discipline, and subsequently, ESP JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9290 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 90 instructors can design exercises requiring learners to identify these justifications that frequently incorporate infinitive clauses used as reason/purpose adjuncts, and integral and/or non-integral citations employed in advocating sampling techniques. These language mechanisms can be manipulated by second language learners to demonstrate comparability of the treatment groups and to minimise possible doubts about the acceptability of sampling criteria involved in experimental research. Acknowledgements I would like to thank the Malaysian University of Sabah for a research grant needed to conduct this study. I am also indebted to the Fulbright Organization for making it possible for me to access a large number of references related to this paper during my research stint at the University of Michigan in ann arbor. [Paper received June 2010] [Revised paper accepted October 2010] References DELINEaTINg SaMPLINg PrOcEDUrES Ibérica 21 (2011): 71-92 91 Berkenkotter, C. (2009). “A case for historical “wide-angle” genre analysis: A personal retrospective”. Ibérica 18: 10-21. Berkenkotter, C. & T.N. Huckin (1995). Genre Knowledge in Disciplinary Communication: Cognition/Culture/Power. Hillsdale, NJ: Lawrence Erlbaum. Bhatia, V.K. (1993). Analyzing Genre: Language Use in Professional Settings. Essex: Longman. Bhatia, V.K. (1995). “Genre-mixing in professional communication: The case of ‘private intentions’ v. ‘socially recognised purposes’” in P. Bruthiaux, T. Boswood, & B. Du-Babcock (eds.), Explorations in English for Professional Communication, 1-19. Hong Kong: City University of Hong Kong. Collins, L. (2007). “L1 differences and L2 similarities: Teaching verb tenses in English”. ELT Journal 61: 295-303. Connor, U. & A. Mauranen (1999). “Linguistic analysis of grant proposals”. English for Specific Purposes 18: 47-62. Connor, U., K. Davis & T. de Rycker (1995). “Correctness and clarity in applying for overseas jobs: A cross-cultural analysis of U.S. and Flemish applications”. Text 15: 457-475. Ding, H. (2007). “Genre analysis of personal statements: Analysis of moves in application essays to medical and dental schools”. English for Specific Purposes 26: 368-392. Döpke, S. (1999). “Cross-linguistic influences on the placement of negation and modal particles in simultaneous bilingualism”. Languages Sciences 21: 143-175. Ellis, R. (2006). “Modelling learning difficulty and second language proficiency: The differential contributions of implicit and explicit knowledge”. Applied Linguistics 27: 431-463. Flowerdew, L. (2005). “An integration of corpus- based and genre-based approaches to text analysis in EAP/ESP: Countering criticisms against corpus-based methodologies”. English for Specific Purposes 24: 321-332. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 91 Dr. Jason Miin-Hwa Lim is associate Professor of English at the Malaysian University of Sabah. His recent publications include papers on ELT and ESP in System (Elsevier), JEAP (Elsevier) AJELT (Hong Kong), and ESPJ (Elsevier). He was also a Fulbright research Scholar at the University of Michigan in 2009 and 2010. JaSON MIIN-HWa LIM Ibérica 21 (2011): 71-9292 Gay, L.R., G.E. Mills & P. Airasian (2009). Educational Research: Competencies for Analysis and Applications (9th ed). London: Pearson Education. Greenbaum, S. & R. Quirk (1990). A Student’s Grammar of the English Language. Harlow, Essex: Pearson Education. Holmes, R. (1997). “Genre analysis, and the social sciences: An investigation of the structure of research article discussion sections in three disciplines”. English for Specific Purposes 16: 321- 337. Hudson, T. (2007). Teaching Second Language Writing. Oxford: Oxford University Press. Hunt, K.W. (1970). “Syntactic maturity in schoolchildren and adults”. Monographs of the Society for Research in Child Development 35, Serial no. 134: 1-67. Kanoksilapatham, B. (2005). “Rhetorical structure of biochemistry research articles”. English for Specific Purposes 24: 269-292. Lim, J.M.H. (2006). “Method sections of management research articles: A pedagogically motivated qualitative study”. English for Specific Purposes 25: 282-309. Lim, J.M.H. (2007). “Crosslinguistic influence versus intralingual interference: A pedagogically motivated investigation into the acquisition of the present perfect”. System 35: 368-387. Lim, J.M.H. (2008). “Analysing recommendations for future research: An investigation into a hybrid sub-genre” in R. Wilkinson & V. Zegers (eds.), Realizing Content and Language Integration in Higher Education, 131-154. Maastricht: Maastricht University. Lim, J.M.H. (2009). “Rhetorical categories and linguistic mechanisms in describing research conditions: A comparative genre-based investigation into researchers’ choices in Education and Applied Linguistics”. The Open Applied Linguistics Journal 2: 67-85. Mauranen, A. (1993). Cultural Differences in Academic Rhetoric. Frankfurt am Main: Peter Lang. Nwogu, K.N. (1997). “The medical research paper: Structure and functions”. English for Specific Purposes 16: 119-138. Paltridge, B. (2000). “Genre knowledge and teaching professional communication”. IEEE Transactions on Professional Communication 43: 397-401. Pho, P.P. (2009). Research articles in Applied Linguistics and Educational Technology: A corpus- based study of rhetorical moves and authorial Stance. Ph.D. thesis submitted to Monash University, Australia. Posteguillo, S. (1999). “The schematic structure of computer science research articles”. English for Specific Purposes 18: 139-160. Quirk, R., S. Greenbaum, G. Leech & J. Svartvik (1985). A Comprehensive Grammar of the English Language. London: Longman. Sachs, R. & C. Polio (2007). “Learners’ uses of two types of written feedback on a L2 writing revision task”. Studies in Second Language Acquisition 29: 67-100. Samraj, B. (2005). “An exploration of the genre set: Research article abstracts and introductions in two disciplines”. English for Specific Purposes 24: 141-156. Swales, J.M. (1990). Genre Analysis: English in Academic and Research Settings. Cambridge: Cambridge University Press. Swales, J.M. (2004). Research Genres: Exploration and Applications. Cambridge: Cambridge University Press. Thomas, S. & T.P. Hawes (1994). “Reporting verbs in medical journal articles”. English for Specific Purposes 13: 129-148. 04 IBERICA 21.qxp:Iberica 13 10/03/11 17:20 Página 92