RESEARCH METHODS IN LANGUAGE STUDY Introduction to Research … (Muhartoyo) 11 INTRODUCTION TO RESEARCH METHODOLOGIES IN LANGUAGE STUDIES Muhartoyo English Department, BINUS UNIVERSITY Kampus Kijang, Jl. Kemanggisan Ilir III No. 45, Kemanggisan/Palmerah Jakarta Barat 11480, ymuhartoyo@yahoo.com ABSTRACT Language research is an area of interest for many students and lecturers of Faculty of Letters. This article is an attempt to describe various research methodologies in language studies in a simple way. The research methodologies covered include experimental research, quasi experimental research, ethnography, and case study. The different concept of qualitative and quantitative research is also highlighted. The issues of reliability and validity of a research report are briefly discussed. Keywords: experimental research, ethnography, case study ABSTRAK Penelitian bahasa merupakan bidang yang menarik bagi mahasiswa dan pengajar di Fakultas Sastra. Artikel ini mencoba menggambarkan berbagai metodologi riset dalam bidang bahasa dengan cara yang sederhana. Metodologi riset ini mencakup experimental research, quasi experimental research, etnografi, dan studi kasus. Artikel ini juga membahas konsep metode riset kuantitatif dan kualitatif. Masalah validitas dan keabsahan sebuah laporan riset dibahas secara singkat. Kata kunci: penelitian eksperimental, etnografi, studi kasus Jurnal LINGUA CULTURA Vol.1 No.1 Mei 2007: 11-18 12 INTRODUCTION There is a felt need for students of Faculty of Letters to get a better understanding on various research methodologies in language studies. Exposing the students with various research methods will enable them to choose an appropriate research method for their final research project. The various methodologies that will be briefly discussed in this article cover experimental method including the logic of inferential statistics, ethnography, and case study. Before discussing these methods, it is worth looking at the issues of binary distinction of qualitative and quantitative research. Although some linguists say that the binary distinction of qualitative and quantitative research is simplistic and naive, the writer agrees with David Nunan‟s (1994) argument that the distinction is real, not an ostensible one. Indeed, qualitative and quantitative research methods are guided by two quite different conceptions. Quantitative research is using a deductive method. It begins with a hypothesis or theory then searches for evidence either to support or to refute that hypothesis or theory. The data collected for this type of research is called quantitative data i.e. data which are recorded in numerical form. Statistical tools such as Correlation and Regression, Wilcoxon Signed Rank Test, Chi-Square, T-test, Analysis of Variance (ANOVA), etc. are used to analyze the data. Other characteristics of quantitative research are obtrusive and controlled, objective, and generalisable. Obtrusive and controlled means that the researcher does some intervention/ treatment to the subjects that they are researching on. For instance, a researcher introduces a new writing method to the subject then the researcher will see whether the new method really works. The researcher will give pretest (before treatment/ intervention) and post test (after treatment/ intervention) to see the effect of the new method. Objective means the data collected are the result of an objective measurement/ evaluation. While generalisable means that the result can be applied or generalized to the population (beyond the sample). Qualitative research, on the other hand, is using an inductive method that seeks to draw general principles, theories or „truth‟ from an investigation and documentation. It does not use any statistical tool, however, simple tabulation and calculation are sometimes used in the analysis. The data collected for this research are normal qualitative data i.e. data which are recorded in non- numerical form, such as transcript of an interview. Other characteristics of qualitative methods are subjective, naturalistic and uncontrolled, exploratory, and descriptive. All knowledge, according to qualitative research, is relative and has a subjective element, consequently, holistic and ungeneralisable studies are justifiable. Other important issues in language studies are dealing with reliability and validity. DISCUSSION Reliability and Validity Important issues that must be addressed by a language researcher in conducting a research are reliability and validity. Reliability means (a) the ability of an independent researcher to reach the same conclusion after analyzing one‟s data and (b) the possibility of similar results to be achieved by replication of one‟s study. A research will have a high degree of reliability if consistent results can be obtained by an independent researcher by reproducing the research using similar methods and procedures. On the other hand, a research has low degree of reliability when an independent researcher conducting similar research with the same method and procedures results in different conclusions. Introduction to Research … (Muhartoyo) 13 Another issue is validity which means (a) the degree of a researchers‟ honesty that he/she has really observed what he/she has to observe in his/her research and (b) the extent of generalization from a researcher‟s finding based on his/her subjects and situations to other subjects and situations. In other words, a research will have high degree of validity if an independent researcher can prove/see that a concerned researcher has really observed what he/she claimed to have been observed during the research and the research result can be generalized beyond the samples and situation of the research. Validity is divided into two different types, i.e. internal validity and external validity. Internal validity means the extent to which a researcher can claim that any differences in research results are due to the treatments given to the subjects. For example, if a researcher can prove that the better test scores of the subjects under his/ her research are result of his/ her treatments/ intervention, it means the research has a high internal validity. On the other hand, external validity is the extent of generalization that can be drawn from samples to populations. It is the challenge of a researcher to achieve research results which have a high degree of reliability and validity. A true language scientist will address these two issues properly in his/ her research. Experimental Research Experimental research is a research method usually used to find out the strength of relationship between variables. In order to be able to use this research method properly we need to know about variables, population, and sample. A variable means anything which does not remain constant, for example, language proficiency, aptitude, motivation, skill, interest, and so on. Variables can be classified into two categories, i.e. independent and dependent variables. An independent variable is an element or item used by the researcher to influence the other variable, for instance a teaching method. A dependent variable is an element or item that is influenced or affected by an independent variable, for example the test scores of students under investigation. Variables can also be classified based on the type of scale used for measuring them. In this classification, variables are classified into 4 groups, i.e. 1) Nominal scale, 2) Ordinal scale, 3) Interval scale, and 4) Ratio scale. A nominal scale is for mutually exclusive characteristics, such as sex and eye color. A subject cannot be simultaneously categorized into „male‟ and „female‟, or „blue-eyed‟ and „brown-eyed‟. An ordinal scale is for variables that can be given a ranking, such as first, second, third. In this case the actual score itself is not given. An interval scale provides information on the ranking as well as the distance between scores. Most test score data belong to this category. Finally, a ratio scale is for absolute value, such as temperature. Applied linguistics is not quite interested in this type of variable as most variables do not have absolute values. After understanding what variables are, we should know about populations and samples. A population is all cases, situations, or individuals who have one or more similar characteristics. For example, seventh semester students of a faculty of letters who have passed a scientific writing course. Meanwhile a sample is a subset of individuals or cases taken from a population. A sample is needed when a population of an experimental research is too big as it will be very tedious and time-consuming to do a research on the whole population. Different techniques of sampling can be seen in books on statistics for research. Example of an Experimental Research Supposing one of students of a Faculty of Letters who has been teaching English in a private school will write a thesis entitled “Innovative English teaching materials for senior high school students.” She has to prove that her innovative teaching materials are really superior to the traditional Jurnal LINGUA CULTURA Vol.1 No.1 Mei 2007: 11-18 14 ones. To do this, she has to select two groups of students, one group that has used the innovative materials (experimental group) and the other group that uses traditional ones (the control group). The experimental group will be taught by using her innovative teaching materials for one term. On the other hand, the control group will get instruction using a traditional English teaching material. At the end of the term, both groups will be tested. However, it is not recommended to test the two groups at the end of the term only as the internal validity of this one-shot test is low. It implies that if the test results show that the experimental group has higher scores than the control group, people will question the method of selecting the groups. It may happen that the members of the experimental group are high achievers while the members of the control group are slow learners. To overcome the threat of internal validity, she has to do a „true experimental research‟ in that she has to select the members of the groups randomly and test the two groups before and after the terms to make sure that the members of the two groups have the same capability and start with equal position. The test must be conducted before and at the end of the term to see the differences made by the two groups. To compare the result of the groups (experimental and control groups), a researcher can use T- test, which is normally used to compare the means of two groups. T-test is used in inferential statistics. By doing a true experimental research she now has a better position to argue that any differences at the end of the terms are caused by the experimental treatment (the use of innovative materials). However, in reality it is difficult to do a true experimental research. It is almost impossible to rearrange the existing class arrangement for experimental research purposes. A researcher often has to accept the intact group of subjects who have been grouped by a school. Although the internal validity will be weakened, this kind of research (experimental research without rearranging the subjects) is still considered to be desirable. This type of research is called quasi-or pre-experimental research. The Logic of Inferential Statistics Experimental research usually uses statistical methods in analyzing research data. Statistical methods can be grouped into parametric and non-parametric statistics (Wijaya, 2000). When the data has abnormal distribution, they have to be analyzed using non-parametric statistical method. However, if the data are normally distributed they can be analyzed by using inferential statistics. There are many interesting features that can be drawn from the data with normal distribution. The data are considered to have normal distribution when the data are equally divided in the distribution chart. It implies that the areas on the left and right sites of the mean are equal (see Figure 1). Figure 1 Percentage of Scores Falling Within 1, 2, and 3 Standard Deviation of the Mean (Source: Nunan, 1994) Introduction to Research … (Muhartoyo) 15 Before discussing the inferential statistics, we need to know such terms as mean, variance, standard deviation, and standard error. Mean is the average of a set of scores (obtained by adding the individual scores together and dividing by the total number of scores. Mean gives information about the central tendency of the scores. Variance is calculated by deducting individual score with the mean and squaring the resulting figures to get rid of the minus signs, adding these together, and dividing by the number of scores minus one. The square root of this figure (variance) is called standard deviation. So standard deviation is a measure of the dispersion of a set of scores from the mean of the score. While Standard error is standard deviation divided by square root of observation in the sample. The standard error is closely related with the level of confindence of the research results. The lower the standard error, the higher the level of confidence of the research result will be. For example, if the standard deviation of the experimental group is 3.8 and the number of students (subjects) is 50, the square root of 50 is about 7, then the standard error is 0.5428 (3.8 divided by 7). It means the level of confidence of this research is 95% which is quite high. To make this discussion about mean, variance, and standard deviation clear, let us see the example in Figure 2 below. As mentioned earlier, inferential statistics can be used to draw many interesting features from a set of data with normal distribution. However, before drawing the interesting features, we have to prove that the data has normal distribution. One of the methods used for proving the normal distribution of the data is Chi Square (Sugiyono, 2005). When the data has been proven to have normal distribution we can make some interesting assumpti-ons. For example, we can see from the Figure 1 that 68% of the scores will be within standard deviation (SD) 1 of the mean, 95% of the scores will be within 2 SDs of the mean, and 99% of the scores will be within 3 SDs of the mean. This implies that students who fall in standard deviation 3 above the mean are the top 1% students, meaning they are the best students. To make it clear let‟s see the following example. A student obtained a raw score of 90 in Grammar and 80 in Writing. Looking at the raw score we may be tempted to draw a conclusion that the student is better in Grammar than in Writing. However, we are not allowed to draw such conclusion by using raw scores. We should know the mean and standard deviation (SD) of the sample before making inferences. Supposed the mean and SD on the test of Grammar are 60 and 15, and in Writing test, 65 and 5, we can conclude that the student is actually better in Writing than in Grammar. Why is 80 better than 90? What is the logic behind this? The reason is as follows. Based on the characteristics of the normal distribution, a score of 90 in Grammar is 2 SDs above the mean [(90-60)/15=2)], meaning that the student is in the top 2.5% of the sample see Figure 1. While the score 80 in Writing is 3 SDs above the mean [(80- 65)/5=3)], meaning that the student is in the top 1% of the sample for that subject. So it is interesting to see that according to the logic of inferential statistics the higher raw score does not necessarily indicate that a student perform better. Score Score-mean² Square of figures in 2nd column 2 -2 4 8 4 16 5 1 1 3 -1 1 4 0 0 1 -3 9 2 -2 4 3 -1 1 5 1 1 7 3 9 40 46 The Sum of the squares/(no. of scores minus 1) = 46/(10-1) = 5.111 Variance = 5.111 Standard deviation (i.e., square root of variance) = 2.26 The Mean = 4. This is calculated by dividing the sum of the scores (40) by the total number of individual scores (10) Figure 2 Calculating Mean, Variance, and Standard Deviation (Source: Nunan, 1994) Jurnal LINGUA CULTURA Vol.1 No.1 Mei 2007: 11-18 16 Ethnography Ethnography is defined as the study of culture/characteristics of a group in real-world rather than in laboratory settings. The researchers do not make any isolation or manipulation to the phenomena that they investigate. According to Wilson (1982) the roots of ethnography were in anthropology and sociology. Wilson relates the tradition to two sets of hypotheses about human behavior. These are the naturalistic ecological hypothesis and qualitative-phenomenological hypothesis. The naturalistic-ecological perspective has the belief that the context in which behavior occurs has a significant influence on that behavior. It means that if we want to find out about behavior, we need to investigate it in the natural contexts in which it occurs, rather than in the experimental laboratory. A research conducted by Bellack, Hersen, and Turner (1978) found that subjects performed in a role-play situation very differently from the way they performed in real-life social situation where the same behavior observed. The qualitative-phenomenological hypothesis means that ethnographers believe that human behavior cannot be understood without incorporating into the research the subjective perceptions and the belief systems of those involved in research, both as researchers and subjects. It implies that the traditional stance of the researcher as „objective‟ observer is inadequate, and the procedures of experimental method of framing hypotheses and operationalising constructs (psychological attributes such as intelligence, aptitude, or motivation which are created to account for observable behavior) before engaging in any data collection or analysis are at best inappropriate and at worst irrelevant. Watson-Gegeo and Ulichny (1988) identify several key principles of ethnographic research. These include the adoption of a grounded approach to data. Grounded approach means the practice of deriving theory from data rather than the other way around. They point out that ethnography involves interpretation, analysis, and explanation – not just description. Proponents of quantitative research criticize ethnographic methods. Their criticisms are concerning the reliability and validity of such research as ethnographies are based on detailed description and analysis of a particular context or situation. However, the problem of internal and external validity will depend on the scope of the research and the researcher‟s purpose. There won‟t be any issue of internal validity if the purpose of research is not trying to establish causal relationship between variables. If the researcher‟s purpose is not seeking generalization his/ her findings, then there won‟t be any issue of external validity. So ethnography is a valid research tradition in its own right, and should not merely be regarded as a device for generating a hypothesis for experimental research. Case Study Linguists have differing opinion regarding the definition of a case study. Some linguists say that a case study is just pre-experimental research, while some others argue that a case study is a legitimate research method in its own right. According to Adelman et. al. a case study is the study of an „instance in action.‟ In this case, a researcher selects an instance (a case) from the class of objects and phenomena, for example, „a second language learner‟ or „a grammar classroom‟ and observes the way this instance operates or functions in context. The concept of studying phenomena in context implies that the case study has similar philosophy, methods, and concern with ethnography. Therefore some linguists tend to say that the case study is a limited type of ethnography. Another difference between ethnography and a case study is in the focus of research in which ethnography is more concerned with the cultural context and cultural interpretation of the phenomena being researched. Introduction to Research … (Muhartoyo) 17 Typically a case study researcher investigates the characteristics of an individual unit – a child, a clique, a class, a school, or a community. The objectives of such investigation are to find out and analyze intensity of the various phenomena in the investigated unit in order to establish generalizations about the wider population to which the unit belongs. According to Adelman et. al. (1976) there are six advantages of using the case study. Firstly, it is „strong in reality‟ and therefore likely to appeal to practitioners, who will be able to identify the issues and concerns raised. Secondly, the claim that one can generalize from a case about an instance, or from an instance to a class. Thirdly, it can represent a multiplicity of viewpoints, and can offer support to alternative interpretations. It can also provide a database of materials which may be reinterpreted by future researchers. Fifthly, the insights of yielded by case studies can be put to immediate use for a variety of purposes, including staff development, within institution feedback, formative evaluation, and educational policy making. Finally, case study data are usually more accessible than conventional research reports, and therefore capable of serving multiple audiences. There are two points of view on the issue of validity of case studies. The first one argues that, while internal validity is important, external validity may be irrelevant. The researchers holding the second point of view say that tests of validity should be as stringently applied to the case study as to any other type of research. Although validity and reliability issues may appear in a case study, it has a great potential to be a research method in applied linguistics. A major strength of it is that a case study can be carried out by graduate students and/ or classroom practitioners who prefers to conduct a small- scale investigations. CONCLUSION There are four different research methodologies that can be used by students in conducting a research in language studies, they are i) experimental research, ii) quasi or pre-experimental research, iii) ethnography, iv) case study. The four methods briefly discussed in this article can enrich students with different tools for conducting language research. Due to limited space, only important aspects of the four research methodologies are given. Additional information from other relevant sources needs to be used in order to get a full view of the four methodologies in language studies. In addition to the four research methodologies, the concept of qualitative and quantitative research methods need to be correctly understood as the two concepts have different underlying approach. It is not relevant to decide whether one method is better than other methods. The important thing is that a student should know which method he/ she can use to suit his/ her research project. Better understanding on the research methodologies in language studies will make students feel more confident to do their research in language. With clear concept and guidance from competent tutors, students will be able to produce high quality research. Jurnal LINGUA CULTURA Vol.1 No.1 Mei 2007: 11-18 18 REFERENCES Adelman, C.D. Jenkins and S. Kemis. 1976. “Rethinking Case Study: Notes from the Second Cambridge Conference.” Cambridge Journal of Education, 6,3,139-150. Bellack, A. S., M. Hersen, and S.M. Turner. 1978. “Role Play Tests for Assessing Social Skills: Are They Valid?” Behavior Theraphy, 9,448-461. Nunan, D. 1994. Research Methods in Language Learning. New York: the Press Syndicate of the University of Cambridge. Sugiyono. 2005. Statistika untuk Penelitian. Bandung: Alfabeta. Wijaya. 2000. Statistika Non-parametrik (Aplikasi Program SPSS). Bandung: Alfabeta Wilson, S. 1982. “The Use of Ethnographic Techniques in Educational Research.” Review of Eductional Research, 47,1,245-265. Watson-Gegeo, K. and P. Ulichny. 1988. “Ethnographic Inquiry into Second Language Acquisition and Instruction.” University of Hawaii Working Papers in ESL, 7,2.