LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 LLT Journal: A Journal on Language and Language Teaching http://e-journal.usd.ac.id/index.php/LLT Sanata Dharma University, Yogyakarta, Indonesia 126 EVALUATING SAUDI EFL SECONDARY SCHOOLS STUDENTS' PERFORMANCE ON PAUL NATION'S STANDARDIZED VOCABULARY LEVEL TESTS Ali Albashir Mohammed Alhaj, Mesfer Ahmed Mesfer Alwadai, and Mohammed Hamed Albuhairi King Khalid University, Saudi Arabia dr_abomathani@yahoo.com DOI: doi.org/10.24071/llt.2019.220112 received 10 January 2019; revised 21 February 2019; accepted 30 March 2019 Abstract The current study aims to evaluate Saudi EFL Secondary School students' performance on Paul Nation's Standardized vocabulary level tests. It also aims to examine the vocabulary size of EFL Saudi Secondary School students at Asir district. Two tools were used for collecting data. The first one was Paul Nation's Vocabulary Levels Test (VLT). The second tool was a questionnaire (TVLSQ) with 13 items. The purposive sample consisted of (80) students besides (20) English teachers. The study sample was selected from Abha secondary schools schools/Saudi Arabia. The study came out with the following results: Saudi EFL Secondary school students' vocabulary size word test falls within 1,000 frequency level, indicating that students are unable to gasp an average text. Saudi EFL Secondary School student who obtained higher marks in 1,000 word level also obtained higher marks in 2,000 –word level and vice versa. These results confirmed the high correlation between the two tests which hits 0.76. Moreover, in Paul Nation's Vocabulary Levels Test (VLT) the more words Saudi EFL Secondary school student knows, the better score his/her four skills. To most secondary school teachers as well as students, English grammar is the main focus on their teaching and learning, not the vocabulary. Keywords: evaluating, performance, standardized, vocabulary, tests/Saudi EFL/secondary school/students Introduction Saudi secondary school students are taught a large number of words but some of them might not be useful to qualify them to perform well on vocabulary standardized tests such as VLT. This might cause a serious problem that influences other skills of English language such as reading, speaking, listening and writing. Moreover, there are other vocabulary aspects that learners have to master to give them a better chance to understand and use language effectively. These aspects include polysemy, connotation, associations and other lexical relations. Consequently, vocabulary learning is not a mere listing of words meaning but more complicated and problematic area of language learning. This research aims to examine Saudi Secondary School students' vocabulary in accordance with Paul Nation's Vocabulary Levels Test (VLT) and evaluate whether the vocabulary taught LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 127 is of high or low frequency. To meet the foregoing stated objectives the following research question is raised to find out to what extent is the initial level of Saudi EFL Secondary School Students' vocabulary size lower than 1000 words as judged and assessed by VLT. This study is supposed to test the following hypothesis: the initial level of Saudi Secondary School Students' vocabulary size is lower than 1000 words as judged and assessed by VLT. This study is important for tests, textbooks designers and writers as it makes them pay attention to choose of the most widely used words in our everyday life and avoid stuffing teaching materials with useless passive words. It also draws educators’ attention to what goes around them in the world in vocabulary teaching, acquisition, learning and updating of high frequency words. The study is also important for test and text designers as well as for vocabulary acquisition researchers: for the former because they are better informed to create materials and tests suitable for different levels and educational needs, and for the latter, because empirical data from different groups of subjects can provide a baseline for comparison and help to identify patterns of vocabulary acquisition and development. The study will also be conducted at some Abha Secondary Schools during the academic year 1439/1440 H (2018/2019). The number of subjects in this study is not large enough. They are students from the different schools, it cannot be thought the same results can be obtained else in Saudi Arabia. It needs for further studies in more secondary schools in other cities or provinces of Saudi Arabia. Finally, some assumptive for vocabulary learning and teaching are suggested. Future study of this kid is hoped to overcome the limitations of this study. Words are essential to communication. Little children learn to speak in isolated words and then in chains of nouns and verbs. The child who says “Daddy bye-bye ear” is easily understood by English–speaking adults. We expect students of a second language, however, to control the grammatical feature of that language as well as its vocabulary. Students are idiosyncratic in the way they remember vocabulary – no two students are exactly the same. In particular, as students become more advanced, their individual interest and needs will help determine what kinds of words they will want to understand, remember and use. Students who are immersed into a new linguistic setting tend to pick up vocabulary first, and then gradually develop a more accurate structural framework in which to use these words. ESL students in the United States of America will, therefore, concentrate on vocabulary acquisition and may use pidgin English before mastering standard English. English–speaking students of French or Spanish, on the other hand, will usually go through a course of instruction that focuses at first on mastery of the sound system and selected structures and subsequently stresses more intensive vocabulary work. Selecting the Words to be Taught Even the best teachers in the most ideal settings can never teach all of the words that learners need. Choices need to be made, when deciding which words to address, teachers will find it useful to distinguish between increasing vocabulary. (introducing new words) and establishing vocabulary (building on and LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 128 strengthening partial word knowledge). Both are important. Paul Nation (1990) argues that'' old material in any lesson is the most important ''(p.7), in part because of the incremental nature of word learning. It is wise to build on initial investment in word learning by giving known words more attention and giving learners a chance to focus on new aspect of a word in contextualize settings. Therefore, teachers will want to select words that their students have seen before but may not be able to use in their own production. They will also want to select new words that students will need. These choices should be guided by students' needs, including their level of learning and their academic and professional goals. (Paul Nation, p.8) The most important words in any language are those that are most frequent. The GSL, for example, is a high-frequency list made up of the 2000 word family that occur most frequently in a variety of domains (conversations, newspapers, novel, news programs, etc.); these high-frequency words make up at least 80% of written texts and 90% of conversation (Nation, 2008). Some frequency lists are designed for specific groups of learners. For example, the Academic Word List (AWL) (Coxhaed's Academic Word list) Retrieved 12/12/2018 http://www.cal.org/create/conferences/20. contains approximately 570 word families that occur most frequency in academic material across four academic domains (business, the humanities, law, and the physical and life sciences). It is based on a corpus of 3.5 million words and is designed to guide word selection for general academic preparation in all fields. The criteria for inclusions on this list were that the wards not appear on the GSL and that they occur frequently and uniformly across the four domains. The AWL is used widely in dictionaries and course materials (see Coxhead, 2000). Other frequency lists identify technical words (words frequently used in specific fields such as science, medicine, or math) (Zimmerman, 2009, p.117). Regardless of the specific target language and the condition of instruction, vocabulary is an important factor in all language learning and teaching. Students must continually be learning words as they learn structures and as they practices the sounds system. This study aims at examining Saudi Secondary School students' vocabulary in accordance with Paul Nation's Vocabulary Levels Test (VLT), as well as evaluating whether the vocabulary taught is of high or low frequency. Method Subject There are two groups of subjects in this research: (1) Saudi Secondary School students answering Nation's tests (VLT), (2) Saudi male and female Teachers of English answering the questionnaire. The participate sample selected for this study consisted of Saudi secondary students at Abha Secondary School complex. The purposive sample consisted of (50) students besides (20) English teachers. In total, 50 students returned completed tests (out of 80 approached, 20 of the respondents were male with the remaining 30 female students). Students aged between 16 and 19. The participate sample selected for this study consisted of Saudi secondary Teachers of English at Abha Secondary Schools. In total 20 teachers of English returned completed questionnaires (out of LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 129 30 approached). This purposive population of the research consisted exclusively of teachers of English who taught English language more than five years. Instruments The present study used two tools. The first one was Nation's standardized Vocabulary Level Test (VLT). The second tool was and the Teacher Vocabulary Learning Strategies Questionnaire. Meanwhile VLT was used to measure students' word level size. Additionally, TVLSQ was used to collect English teachers ' overall opinions on vocabulary learning. Questionnaire The second tool of gathering data was the Teacher Vocabulary Learning Strategies Questionnaire which was adapted to teachers' experience, for example, with regards to the motivational aspects of English vocabulary learning, teachers were asked about their students' knowledge of vocabulary and motivation. The remaining sections, deal with linguistic aspects, English vocabulary teaching, and vocabulary teaching assessment. A. Validity To test the validity of the questionnaire, the questionnaire was pre-piloted first with some experts in the field, four assistant professors at King Khalid University, Saudi Arabia. A thirteenth item was added to the third (Evaluation of lexical competence), which was' Paul Nation's Vocabulary Level Test is unique to assess my students' vocabulary size). Also, a phrase was added to the first item, outside classroom. B. Reliability of the Questionnaire First, all the items included in The TVLSQ were checked for their reliability, Cronbach's alpha is well-known coefficient that estimates the proportion of variance that is systematic or consistent in a set of test scores. The Cronbach's alpha scores (n=20) obtained for this study instruments were of 0,654, which can be considered a good indicator of its reliability. The major sub- scales obtained the following scores. Table 1. Reliability Coefficient of TVLSQ Q. Pivots Motivational Aspects Linguistic Aspects Evaluation of Lexical Competence Total scores Reliability Coefficient 0.742 0.778 0.668 0.654 Statistical dif. 0,01 0,01 0,01 0,01 It can be seen from table (1) that TVLSQ has a higher degree of consistency reliability. Procedure Students' Vocabulary Level Test Permission to undertake the research was first obtained from the principal of Abha Secondary School and then the head department of English Language. The total number of test copies distributed by the volunteer teachers was 80. The number of the returned valid copies was 50. The time allowed was forty minutes. The researcher, as a shooter trouble, ensured that the rubric of the test was very LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 130 comprehensive and clear to all .The researcher marked the test. The maximum mark of the test was 30. Ethics of scientific research were closely observed throughout the different stages of this study. This included (and are not limited to) respecting students' privacy, preserving and accurately presenting their answers, and not using threatening and/or fraudulent means to force them to participate in the study. Respondents were briefed about the nature and goals of my study in order to obtain their informed consent. English Teachers' Questionnaire As indicated in the previous section, the participants of the questionnaire were teachers of English. There were 30 participants in total since each school of the complex had about 10 teachers of English. The total number of the questionnaire copies distributed by volunteer teachers was 30. The number of the returned valid copies was 20. This high rate of returned papers was attributed to the following reasons: 1. Effort exerted by volunteer teachers, most of them were my colleagues. 2. The investigated subject might be of great interest to respondents. 3. Only one or two sections of the questionnaires were answered. 4. Some participants carelessly answered the items of the questionnaire. 5. Four or more items were unanswered. Thus, the number of valid questionnaire was 20. 6. was used to collect English teachers' overall opinions on vocabulary learning Findings and Discussion Data Analysis Results This section reports the results of the analysis of the data from the Teacher Vocabulary Learning Strategies Questionnaire (TVLSQ) which aimed at gathering data from other side of the learning process. The results were analyzed and discussed in order to answer the following research question: is the initial level of Saudi Secondary School Students' vocabulary size lower than 1000 words as judged and assessed by VLT? The results show that Saudi Secondary School Students' vocabulary size is within 1000 word level. Below are the means and standard deviations, ranges, for students’ scores which show: Table 2. Type of VLT Mean(x) Standard Deviations ( SD) T. Value Function VLT 1,000 words 23.48 7.18 6.65 ,01 VLT 2,000 words 18.32 8.39 Table 2 shows the means and standard deviations for the 1,000 word test and the 2,000 frequency band of the VLT. As can be seen, the mean score for the former is 23.48, whereas for the latter is 18.32. LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 131 As can be also seen from Table 2 the mean scores achieved by Saudi secondary school students of some Abha's schools in both levels. The mean of23.48 points obtained in the 1,000 frequency level drops sharply to 18.32 in 2,000 VLT. The results of the t.test applied to the means of each frequency level gave us the following value T=6.65. This value is significant at the ,01 level. It can be claimed that Saudi secondary school students know considerably fewer words from the 2,000 than from the 1,000 frequency level. The low scores achieve indicate that Saudi secondary school students know English words from the 2,000 frequency band. The correlations were conducted between scores on the 1,000 and 2,000 word frequency levels. Results show a highly correlation between the two tests (R = ,76) In order to ascertain whether there are significant differences between the scores of the students in two tests, their means scores were compared for the 1,000 and 2,000 word level tests. Table 3 presents the means and standard deviations for both group. Table 3. Type of VLT Mean (x) Standard Deviations (SD) VLT 1,000 words 23.48 7.18 VLT 2,000 words 18.32 8.39 As can be seen from table 4 there is statistical difference between means scores of Saudi secondary school students in both tests (VLT 1,000 words=23.24), VLT 2,000 words=18.32). The differences are due to T. value which hits (6.65) (see table 2). These values are significant at the ,01 level. The results confirmed the differences in favor of VLT 1,000 word test. Figure 1. Shows knowledge of words from VLT, 1,000 words was less than that of VLT, 1000 words As can be seen from Figure 1 that the English receptive vocabulary size of Sudanese Secondary school students as measured by the 1,000 word test falls within 1,000 frequency level. However, this does not mean that these students master this level since scores reveal that half of them recognize less than two-third of the words from this level. LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 132 Figure 2. Shows knowledge of words from VLT,2000 words was less than that of VLT, 1,000 words As can be seen from figure 2 that knowledge of words from the second thousand most frequent was less than that of the first one thousand. Figure 3. Compares the results of both tests (VLT, 1,000+ VLT, 2000 words) As can be seen from figure 3 the performance of Saudi Secondary school students in 1,000 –word level was better than in 2,000–word level. Figure 3 indicates that, the student who obtained higher marks in 1,000–word level also obtained higher marks in 2,000–word level and vice versa. These results confirmed the high correlation between the two tests which hits ,76. The Evaluation of Performance of Saudi Secondary School Students in 1000 and 2000 VLT 1. Saudi Secondary school students’ vocabulary size is lower than the requirement of the curriculum. The reason for this is in Saudi secondary school, students have so many classes to learn they cannot spend much time on English. 2. Saudi Secondary school students do not have scientific vocabulary learning strategies. Many students remember words by reciting or copying, and they do not much know about useful vocabulary-learning ways. 3. Teachers and students do not pay more attention to vocabulary learning. To most secondary school teachers, English grammar is the main focus on their teaching, not the vocabulary. Language skills are important in language LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 133 learning, vocabulary is the base of all the language skills, especially for students in secondary education. In this way, English teachers should focus on students' vocabulary learning and they must encourage their students to remember more words in order to improve their vocabulary size and then improve their language skills. 4. The typical way of students learning English is in a direct teaching by teachers in English classes, but in other lessons or not in the English class, English is not widely used. So, students do not have many opportunities to practice English. In addition, it is often limited for Saudi Secondary school students to read English materials, especially those have relation with language comprehension ability. Testing Hypotheses Hypothesis One Hypothesis 1 is stated as follows: the initial level of Saudi Secondary School Students' vocabulary size is lower than 1000 words as judged and assessed by VLT. This hypothesis addresses Research Question 1 (see the introduction) which aims to find out the initial level of Saudi EFL Secondary School Students' vocabulary size which is hypothesized to be lower than 1000 words as judged and assessed by VLT. Consequently, through the data gathered this hypothesis was rejected and it was found that The English receptive vocabulary size of Saudi Secondary school students as measured by the 1,000 word test falls within 1,000 frequency level. According to the instrumentalists’ view, individuals who perform better on a vocabulary test probably understand more of the words in texts they read than individuals who score lower do. The central idea of this hypothesis is straightforward: knowing the words enables reading comprehension. In other words, this position claims that vocabulary knowledge is a direct factor in the causal chain resulting in reading comprehension. The educational implications of the instrumentalist hypothesis are apparent: in order to improve students' reading ability. Vocabulary teaching should be regarded as a priority in the curriculum. The larger a student's vocabulary, the better he or she will understand a text. The Results of Teacher Vocabulary Learning Strategies Questionnaire (TVLSQ) 1. More than 0,95= the achievement of statements in real context. 2. Less than 0,64= no achievement of statements in real context. 3. When the statement comes between 0,64 and 0,95 shows that achievement of statements in average level. Conclusion The English receptive vocabulary size of Saudi Secondary school students as measured by the 1,000 word test falls within 1,000 frequency level. The student who obtained higher marks in 1,000–word level also obtained higher marks in 2,000 –word level and vice versa. These results confirmed the high correlation between the two tests which hits ,76. There is statistical difference between means scores of Saudi secondary school students in both tests (VLT 1,000 words=23.24), VLT 2,000 words=18.32).The differences are due to T.value which hits (6.65), 1,000 word level was better than in 2,000–word level. The results show that Saudi LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 134 Secondary school students 'vocabulary size is within the 1000 and 2000 word level. Their vocabulary knowledge is relatively low at the 2000 word level, indicating that they will struggle to understand an average text. The results revealed that receptive of words in the 1000 and 2000 frequency levels plays a role in assessing reading, speaking, listening, writing quality. In this sense, the more words Saudi Secondary school student knows, the better score his/her four skills. Many words students do not know the meaning of these words. That is to say, the percentage of the words students remembering is really low. It tells us Saudi Secondary school students’ vocabulary size in 2000 level test is so limited. Numerous studies indicate that vocabulary knowledge is an important factor for understanding when reading, writing text. Therefore, students’ vocabulary assessment and instruction should be a focus of every English course especially at the secondary level in order to prevent the Matthew effect, that poor readers read less and acquire fewer words, while better readers read more and learn more words from their reading. To conclude vocabulary is central to language and of great importance to the language learners. Without the mediation of vocabulary and lexical knowledge, basic communication would be impossible even with grammatical or the other type of linguistic knowledge. It is considered by both first and second researchers to be of great importance in language competence. It is vital to motivate students to rely on textual and contextual clues in dealing with unfamiliar words rather than looking them up in a bilingual dictionary. The results of the current research call for future research on surveying the relationship between receptive vocabulary, written skills and individual differences. The present study might also be useful, with some modifications, to be carried out in the other secondary schools in Sudan to see if there are differences between students' performance in each secondary school. More research on Saudi Secondary school students' vocabulary and teaching is required. References Abu-seif, L. (1998). Developing skills in English among non-specialist graduate students. Unpublished PhD thesis. Faculty of Education, Aim Shams University. Allen, V. F. (2003). Techniques in teaching vocabulary. Oxford: Oxford University Press. Al-Samawi, A. (1987). A proposal to improve classroom interaction of Yemeni teachers of English through teachers training program on classroom interaction analysis. Unpublished M.A. thesis: Moray College of Education, Edinburgh UK. Anderson, J. (1981). Assessing reading. Cambridge: Cambridge University Press. Anglin, J, M. (1993). Vocabulary development: A morphological analysis. Monographs of the Society for Research to Child Development, 58(10). Ateya, M. E. H. A. (1996). Suggested course for developing vocabulary learning strategies of students of English in faculties of education. Unpublished PhD thesis. Faculty of Education, University of Zagazig, Benha. Available at (http://dx.doi.org/10.2307/3586733). LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 135 Bachman, L. (2002). Language testing in practice. Oxford: Oxford University Press. Available at: http://www.cambridge.elforg./N Brown, J. D. (2000). Using surveys in language programs. Cambridge: Cambridge University Press. Celce-Murcia, M. (1996). Teaching English as a second or foreign language. Los Angeles: Heinle & Heinle. Cobb, T. & Horst, M. (1999). Vocabulary sizes of some city university students. City University (HK) Journal of Language Studies, 1, 59-68. Corson, D. (1995). Using English words. Dordrecht, Netherlands: Kluwer Academic Publishers. El-Minyawi, R. A. (1984). The development of an ESP material with special reference to students of Home Economics. Unpublished PhD thesis. Faculty of Education, Aim Shams University. Eyckmans, J. (2001). Relationships between productive and vocabulary knowledge of speaking performance of American learners of English at the novice level. Unpublished PHD thesis, Faculty of Education. Arizona University. Gower, R. (1995). Teaching practice handbook (Handbooks for the English classroom). Cambridge: Cambridge University Press. Graves, M. F. (1988). Vocabulary learning and instruction. Review of Research in Education, 13. Hatch, E. (2001). Vocabulary semantics and language education. Cambridge: Cambridge University Press. Haynes, M. & Bader, I. (1993). American and Chinese readers learning from lexical familiarization in English texts. In T. Huchin, M. Haynes, & J. Coady (Eds.), Second language reading and vocabulary acquisition (pp.130-152). Norwood, NJ: Ablex. http://www.cal.org/create/conferences/20. . contains approximately570 http://www.iises.net/international-journal-of-teaching-education/publication- John, S. (2002). Remove limits to learning with systematic vocabulary instruction. Cambridge: Cambridge University Press. Laufer, B. (2001). How much lexis is necessary for reading comprehension? London: Macmillan. Malley, J. M. (1990). Learning strategies in second language acquisition. Cambridge: Cambridge University Press. Moir, J. (2002). Learners’ use of strategies for effective vocabulary learning. Cambridge: Cambridge University Press. Murcia, et al. (2014). English teaching as second or foreign language. Cambridge: Cambridge University Press. Nagy, W. (1997). On the role of context in the first-and second language vocabulary learning. In N. Schmitt (Ed.), Vocabulary: Description, acquisition and pedagogy. Cambridge: Cambridge University Press. Nation, I. S. P. (1993). Vocabulary size, growth, and use. The Bilingual Lexicon, 115-134. Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge: Cambridge University Press. LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 22, No. 1, April 2019 136 Ostyn, P. & Godin, P. (1985). RALEX: An alternative approach to language teaching. Modern Language Journal, 69, 346-353. Pesnick, L. B. (1989). Knowing, learning and instruction. Hillsdale, NJ: Lawrence Erlbaum. Qian, D. D. (2002). Investigating the relationship between vocabulary knowledge and academic reading performance: An assessment perspective. Language Learning, 52(3), 513-536. Read, J. (2004). Research in teaching. Annual Review of Applied Linguistics, 24, 146-161. Richards, J. C. (1976). The role of vocabulary teaching. TESOL Quarterly, 10, 77- 89. Rubin. J. (1981). Study of cognitive processes in second language learning. Applied Linguistics, 2, 117-131. Schmitt, N. (2002). Vocabulary description, acquisition and pedagogy. Cambridge: Cambridge University Press. Seliger, H. W. (2002). Second language research methodology. Oxford: Oxford University Press. Sokmen, A. J. (1997). Current trends in teaching second language vocabulary. Cambridge: Cambridge University Press. Tschirner, E. (2004). Breadth of vocabulary and advanced English study: An empirical investigation. Electronic Journal of Foreign Language Teaching, 9(1). Ulijn, J. M. & Strother, J. B. (1990). The effect of syntactic simplification on reading EST texts as L1 and L2. Journal of Research in Reading, 13, 38-54. van Bruggen, J. A. (1946). Factors affecting regularity of the flow of words during written composition. Journal of Experimental Education, 15(2), 133-155. Vermeer, A. (2002). Breadth and depth of vocabulary in relation to L1/L2 acquisition and frequency of input. Applied Psycholinguistics, 22. Webb, S. (2005). Receptive and productive vocabulary learning: The effects of reading and writing on word knowledge. Studies in Second Language Acquisition, 27, 33-52. www.nmt.edu/-xtnguyen. Weigle, S. (2004). Assessing writing. Cambridge: Cambridge University Press. Zimmerman, C. B. (2009). Word knowledge. New York: Oxford University Press.