Research and Evaluation in Education e-ISSN: 2460-6995 Research and Evaluation in Education Volume 1, Number 2, December 2015 (pages 114-128) Available online at: http://journal.uny.ac.id/index.php/reid THE EFFECTIVENESS OF ENGLISH TEACHING PROGRAM IN SENIOR HIGH SCHOOL: A CASE STUDY 1) Alfred Irambona; 2) Kumaidi 1) Burundi National University, Burundi; 2) Yogyakarta State University, Indonesia 1) irambonaalfred@yahoo.fr; 2) kuma_426@yahoo.com Abstract This article evaluates the effectiveness of English teaching program in the eleventh graders of senior high school. This study was a summative case study using a mixed-method. Descriptive statistics were used to analyze quantitative and qualitative data followed by a descriptive analysis following Context, Input, Process, and Product (CIPP) model. The informants were 43 students of Sekolah Menengah Atas Negeri 3 Yogyakarta (SMAN 3 Yogyakarta) -- 3 Yogyakarta State Senior High School and two English teachers of the school. The findings reveal that the program objectives, classroom condition, students’ needs and barriers are in effective category. In input component, it is found that both teachers are qualified and experienced. The teaching training is not sufficient; students’ textbooks and course designs are in effective category. The process component shows that teaching materials, teaching methods, teaching activities and assessments are in effective category. The product component shows that English marks, students’ needs and barriers are in effective category. However, the teaching materials are in ‘not effective’ category. Keywords: program effectiveness, program objectives, students’ needs and barriers, teaching methods, teaching materials mailto:irambonaalfred@yahoo.fr Research and Evaluation in Education 115 - Volume 1, Number 2, December 2015 Introduction Different countries have different policies. Each country has its own visions for its people to fulfill. To achieve that, respective countries plan how to reach their goals, and the most effective way to pass through is education. Nelson Mandela states, ‘Education is the powerful weapon which you can use to change the world,’ one can understand to which degree education plays a great role in changing a country. His great idea has been concurred with the idea of Moore (2009, p.9) in which he suggests that changes in society are often reflected in more demands being placed on our educational system. It is a luck to have a country whose people are educated. There is no country which can develop without education. Each country needs to make educational changes according to its needs. Through education, people may get knowledge and good behavior which are the best factors leading to the bright life and future. With their knowledge, patriotism, innovations and inventions arise. That is why actually, many governments are doing their best to invest in education, trying to adjust the level of their national education with the standard of the countries who have possessed high standard or best education. They always try to make a new educational program in order to refresh the educational institution in their respective countries according to the world's trend. The Minister of Education and Culture of Indonesia, Nuh (2013) announces that ‘The future of this nation depends on the new curriculum.’ According to his quotation, it is obvious that without changing the education, there will be no progress. Due to that issue, Indonesian government officials, especially those in educational sphere, sat together to elaborate the currently implemented curri- culum, namely Curriculum 2013, and all corresponding teaching/learning materials, its corresponding programs, textbooks, without putting aside how that curriculum is going to be implemented. A curriculum is a vehicle of the govern- ment’ needs in all fields if there is something to change or promote, for instance, languages, arts, and sciences. In line with this, Fterniati and Spinthourakis (2006, p.42) state that: The curriculum makes significant efforts towards promoting instructional change in the way language arts and all subjects are taught. The successful implementation of the new national curriculum needs to be based on appropriately designed materials, continuing in- service instruction of educators as well as informed and supportive school subject advisors to serve as methodology facilitators. Together they create a frame of authentic and more effective praxis which can lead to students who will become engaged, literate and critical citizens in the twenty-first century. However, those materials have to be evaluated before, during, and after the implementation, just to verify whether they match the intended countries’ goals or not. If the countries’ objectives are reached, they can be strengthened. In case they are not attained, decisions on whether to maintain the curri- culum or just to change it can be taken. This situation causes people to observe many curriculum changes in many countries where their results did not correspond to what they were expecting to get, or simply their needs have changed with the time. At this stage, Indonesia, which is the country where this study is carried out, can be a good example to be taken to clarify the case. Time changes and things change accordingly. Since the independence year, namely, 1945, Indonesian curriculum has been changed many times as the following: 1947, 1952, 1964, 1968, 1975, 1984, 1994, 2004, 2006, and finally 2013. It has been changed ten times because the governmental needs changed over the time. The needs maybe not only to catch up with its surrounding countries, but also with the whole world. Now, since July 2013, the government is trying out the new curriculum 2013 to verify if it suits to Indonesian’s needs. To do so, the Ministry of Education chose some pilot schools, including Sekolah Menengah Atas Negeri 3 Yogyakarta (SMAN 3 Yogyakarta) -- 3 Yogyakarta State Senior High School, where the new curriculum was tried out. The results from this trying out will be Research and Evaluation in Education The effectiveness of English teaching program... - 116 Alfred Irambona & Kumaidi the cornerstone to the Ministry of Education and Culture in taking decision about the Curriculum 2013. Since then, many research- es and evaluations have been carried out to analyze the effectiveness or the impact of the implementation of the new curriculum. Evaluation is a crucial activity which has to be done in any domain where people want to know the progress of their program. In educational sphere, evaluation is more than being important because it depicts the effectiveness, strengths, weaknesses, or the failures of a given program. In other words, it shows to which extent students have reached the desired standards. Thus, it is understood that an effective evaluation will be based on that set of objectives which students are intended to meet. These objectives or goals are considered as key guides of program evaluations in educational domain since they clearly mention targets and expectations of education. More- over, they outline materials for grades and what topics to be taught in schools, how topics are sequenced and presented to students, what levels of understanding are to be expected, as well as what skills students will have to develop, and when. In few words, curriculum can be taken as a bridge leading to the wise island of knowledge, where skills are improved. Actually, Indonesian government is working hard to put its students on the level of other countries in the matter of English teaching language in an updated way that can foster students to face the global development in each domain such as economy, culture, science, politics, and technology. Indonesia is now increasingly concerned to produce competitive citizens who will be able to respond positively to a new environment, new world, and new era. It is trying to produce people who can adapt, change, and learn new skills at different points in their lives and who will contribute to the society which they wish to develop in the future. To be successful, the government opted to use communicative approach across the educational levels since 1994, referring to the current trends of English language teaching and changes in the world. Problem Identifications SMAN 3 Yogyakarta is said to be one of the best schools in Indonesia. Its students are said to be among the best students in Indonesia. Before using the new program, Indonesian education employed the one established in 2006. The researchers felt that teachers are not yet ready for the new curriculum because the teachers did not get sufficient training time. Moreover, the books and other teaching facilities such as video tapes, Compact Discs, teaching materials, and teaching aids are not yet ready, and some of the, are also not sufficient. The insufficient facilities can be another barrier to the satisfaction of students’ needs. In addition to that, the researchers wondered whether the actual English pro- gram matches the diverse students’ needs or not. The question of the teaching environ- ment also emerges since it is among the influencing factors in education. Problem Formulations Before conducting this case study, the evaluators were very curious to know if this new Curriculum 2013 is going to be effective or not. Based on this curiosity, the problems questioned in this study are outlined as follows: (1) How is the context of the English Teaching Program at SMAN 3 Yogyakarta in grade eleven?; (2) How is the input of the English Teaching Program at SMAN 3 Yogyakarta in grade eleven?; (3) How does the teaching process of the English Teaching at SMAN 3 Yogyakarta respond to the needs and barriers of the eleventh graders?; (4) How effective the English Teaching Program is at SMAN 3 Yogyakarta in grade eleven? Research Objectives The research objectives can be formulated as the follows: (1) Finding out if the context falls in line with the needs and barriers of the eleventh graders of SMAN 3 Yogyakarta; (2) finding out whether the time allotted to teaching-training related to the Curriculum 2013 was enough or not; (3) finding out if the instructional materials correspond to the teaching English of the Research and Evaluation in Education 117 - Volume 1, Number 2, December 2015 new program; (4) discovering the strategies used to implement the English teaching pro- gram at SMAN 3 Yogyakarta, eleventh grade; and (5) Finding out the effectiveness of the English teachers in their teaching activities at the eleventh grade of SMAN 3 Yogyakarta. Understanding What Evaluation Is Evaluation is defined as a ‘systematic attempt to gather information in order to make judgements or decisions’ (Lynch, 1996, p.2). According to Yarbrough, Shulh, Hopson, et al. (2010, p.xxiv), ‘Evaluation is a systematic investigation of the worth or merit of an object’. If we analyze the precedent definition, it can be concluded that that evaluation is a systematic determination of a subject's merit, worth and significance, using criteria governed by a set of standards. In the book of Evaluation of Educational Program, Fernandez (1984, p.1) says that evaluation is defined as ‘the process of determining to what extent the educational objectives are being realized’. According to Sanders and Sullins (2006, p.1), the term evaluation is understood as ‘the process of systematically determining the quality of a program and how it can be improved’. According to Stufflebeam and Shinkfield (1985, p.159), Evaluation is the process of delineating, obtaining, and providing descriptive and judgmental informa-tion about the worth and merit of some objectives, goals, design, implementation, and impact in order to guide decision making, serve needs for accountability and promote understanding of the involved phenomena. This definition sees evaluation as a well detailed and organized process where the evaluator collects all needed information about the program understudy, analyze them before judging its worth. Another point of view is the one proposed by Kaufman and Thomas (1980, p.4) who define evaluation as ’A process used to assess the quality of what is going on. Evaluation may determine what is working, what to change, and what to keep.’ Based on the different definitions given, it is obvious that all definitions have some- thing in common. All of them converge on judging the value of something, taking decisions, or worthiness of it. This definition leads to the understanding of program evaluation. Program Evaluation Program evaluation is an activity used in any domain where programs are implied. In any sphere where a given program is in use or being implemented, it is a great need to evaluate it in order to know its worth. In any case, policy-makers or stake- holders are curious to know if their programs in use are effective or not, or simply if they are bringing the desired results. In case it works, good and fine, it will be noticed and developed. However, if it does not work, a need of change will be called upon. It is good to mention that a program can be thought of as a group of related activities that is intended to achieve one or several related objectives (Hawthorn & McDavid, 2006, p.15). To meet different expectations of a given country, respective state and federal policymakers have to evaluate their different programs to verify if they are moving in the desired way as planned. Likewise, the present study is going to focus on the evaluation of the English Teaching Program used in senior high schools to see if the new English Teaching Program is effective or not. According to Rossy, Lipsey, and Freeman (2004, p.16), ‘Program evaluation is the use of social research methods to systematically investigate the effectiveness of social pro- grams in ways that are adapted to their political and organizational environments and are designed to inform social action to improve social conditions’. From the quotation, it is understood that program evaluation is the use of different social research methods to check the effectiveness of social programs, find out if the programs have been implemented as planned by the society, organization, or by the government. In other words, we can conclude that a program evaluation is all the systematic steps Research and Evaluation in Education The effectiveness of English teaching program... - 118 Alfred Irambona & Kumaidi passed through in looking for the worth and whether the merit of a given program (the whole program or its different components) is working well as planned or not, looking for different hindrances it is facing, or the pro- gress it is making. Effectiveness After giving a varied set of definitions of a program evaluation, the next question was to know when a given program is said to be effective, something which can be known only through evaluation. Evaluating the effectiveness of a program is not a simple task. A program will be effective when it has achieved its intended objectives/goals. Regarding to the definition of effective- ness, Cambridge Advanced Learner’s Diction- ary (2002) gives an understandable definition. It states that ‘something is effective when it is successful or achieving the results that you want.’ Oxford Dictionary embraces the same idea that ‘effectiveness is the degree to which something is successful in producing a desired result; success’. In the same way, it can be understood that effectiveness is the degree to which objectives are achieved and the extent to which targeted problems are solved. Objectives are needed to identify the expected outcomes, to suggest directions, and to determine the means of evaluation. If one tries to analyze the different definitions given, he/she can draw a conclusion that through objectives, the effectiveness of a teaching program can be deduced from there. In educational institutions, the a teaching pro- gram will be claimed as being effective in case the stated objectives have met the needs of the students. Program effectiveness is a determin- ation of worth made after deliberating about preset planning goals and judgments made during program evaluation processes. In case credible objectives are attained, its effective- ness will be drawn. Otherwise, the program will be claimed as being ineffective. It is fair to say that the effectiveness of a teaching program is based on students’ achievement. No one can dare to say that a given teaching program is effective when students are failed or are just fairly passing with low marks. Thus, can we say that the effectiveness of a program will just happen or bring itself? The answer to that question is NO. If we try to make a deep analysis of the meaning of effectiveness, we will discover that it is a hard work resulting from the combination of many factors. In this study, the researchers did not forget the role of teachers who play a central and vital role in teaching activities as long as they are the ones who make the teaching activities happen, to mean that they are the ones to help the students achieve the stated goals. Moreover, some English exposure, the needs of the students, teaching methods, and teaching/ learning materials were interrelated in discussing about the program effectiveness. Effective Teaching Effective teaching was, is and will be the main dream that any good teacher wishes to reach. According to the IDEA Center Journal (2002, p.1) teaching is effective in case it has an impact on students, for instance, if the students have made progress in their learning activities. In the journal, it is clarified that its main indicators of the effectiveness are derived by answering one question: Do students make progress in achieving objectives selected by the instructor? Thus, it is understood that if the chosen objectives are attained, the teach- ing effectiveness can be claimed. In the same way of thinking, effective teaching can be defined as teaching that successfully achieves the leaning by pupils intended by the teacher (Chris, 2009, p.7). Teaching Methods Any teaching activity without teaching methods is meaningless. There are so many teachers who are intelligent, full of knowledge, whose teaching the outcomes are not satisfying. They fail because they are not equipped with good teaching methods, or they simply do not know how to merge content and delivery process. This means that teaching methods are the first key of teaching success. A good teacher is the one who is not only equipped with a range of teaching http://www.businessdictionary.com/definition/degree.html http://www.businessdictionary.com/definition/objective.html http://www.businessdictionary.com/definition/problem.html Research and Evaluation in Education 119 - Volume 1, Number 2, December 2015 methods, but also who knows how to inter- mingle them according to the material, situation, and learners. Thus, teachers must be trained so that their teaching activities bring desirable outcomes. CIPP Model In this research at hand, the CIPP model which is developed by Stuffbeam (1970) as mentioned by Fernandez (1984, p.7) was used. The acronym of CIPP stands for four types of evaluation functions namely Context, Input, Process, and Product. This model was chosen because of its suitability. In the evaluators’ view, it sounds more effective than the others because it focuses on all stages of the program, whether it is at the beginning, in the middle, or at the end of the program. Zhang, et al. (2011, p.1), says that: ... the CIPP evaluation model is designed to systematically guide both evaluators and stakeholders in posing relevant questions and conducting assessments at the beginning of a project (context and input evaluation), while it is in progress (input and process evaluation), and at its end (product evaluation). In line with that, Stufflebeam and Shinkfield (1985, p.162) explain that CIPP provides information either before or during a project and allows making an evaluation on each of its components (context, input, process, or product) or a combination of its components due to the needs of the audiences. Fundamentally, the use of CIPP model is intended to promote growth and to help the responsible leadership and staff of an institution to obtain and use feedback of systematically so as to excel in meeting important needs, or at least, to do the best they can with the available resources. (Stufflebeam & Shinkfield, 1985, p.166). In short, this method is the one which is mostly used in program evaluations because it has a lot of advantages as long as it gives a general picture of what to focus in every single step. Achieving the effectiveness of a pro- gram is not an easy task. It is a combination of many factors such as different textbooks used in the teaching activities, syllabi, different methods used while teaching, and other teaching facilities. In order to know the effectiveness of the current English Teaching Program, the CIPP model evaluation by Stuffbeam was chosen. This type of model of evaluation consists of four stages namely Context, Input, Process, and Product. The context stage is dealing with the different needs of beneficiaries, who are the students in this case. The goals or program objectives to be attained are also interrelated in this part. Problems or barriers of the needs attainment are focused here. The main point will be the checking out of the availability of teaching materials and their verification to discover if they will help to respond to the students’ needs. The input stage focuses on how teachers plan their lessons. It will be also the time of checking if the selected teaching materials are going to help the different needs of the students. It includes different teaching books which are chosen by teachers to be used and all additional teaching materials. In this section, teachers’ training will also be taken into consideration as one of the strategies which is used to reach the students’ needs. With regard to the process stage, it is about how teachers try to put his/her planned lessons into practice as planned following the schedule for example. Moreover, this section deals with the different strategies and methodologies that teachers will use to reach the needs of their recipients – learners. The research will verify whether the different strategies or the differently-used teaching methodologies really help to answer the students’ needs or not. The product stage is the time of collecting all findings from the different components of the method being used. It is followed by their analysis so that the objectives of the program can be reviewed whether they are met or not, whether the needs of the students are fulfilled or not. Thus, based on the different stages of the CIPP model, its summary is the following. Research and Evaluation in Education The effectiveness of English teaching program... - 120 Alfred Irambona & Kumaidi Research Questions The research questions which guided this study were elaborated following the CIPP Model. Those questions were: Context: (a) What were the English program objectives set in Curriculum 2013?; (b) Up to which degree did the English pro- gram objectives set in Curriculum 2013 cover the students’ needs at the eleventh grade of SMAN 3 Yogyakarta?; (c) Up to which degree did the English program objectives set in Curriculum 2013 cover students’ problems at the eleventh grade of SMAN 3 Yogyakarta?; (d) How conducive was the classroom environment of SMAN 3 Yogyakarta in helping students to meet their needs and overcome their language hindrances? Input: (a) What were the qualifications and experience of the English teachers at the eleventh grade of SMAN 3 Yogyakarta?; (b) What were the perceptions of the English teachers at the eleventh grade of SMAN 3 Yogyakarta about teaching training related to Curriculum 2013?; (c) How was the availabil- ity of teaching/learning materials at the eleventh grade of SMAN 3 Yogyakarta?; (d) How did English teachers at the eleventh grade of SMAN 3 Yogyakarta design their lessons? Process: (a) How were the available teaching/learning materials helping the students to overcome their needs and problems at the eleventh grade of SMAN 3 Yogyakarta?; (b) How effective were the English teaching methods in helping the students of the eleventh grade of SMAN 3 Yogyakarta to overcome their needs and problems while learning English?; (c) In connection with the students’ needs and problems, how were some teaching activities practiced in classes?; (d) What were the perceptions of English teachers of the eleventh grade of SMAN 3 Yogyakarta regarding to the assessments? Product: (a) How were students’ needs and problems dealt with the end of the first semester at SMAN 3 Yogyakarta?; (b) What were the perceptions of the students and their teachers of the eleventh grade of SMAN 3 Yogyakarta regarding to the students’ English marks at the end of the first semester?; (c) What were the perceptions of the students of the eleventh grade of SMAN 3 Yogyakarta regarding to the available teaching/learning materials?; (d) What were the perceptions of the students of the eleventh grade of SMAN 3 Yogyakarta regarding to the implemented English teaching program? Research Methods This study is a summative evaluation of a case study which employed a mixed- method, also called a pragmatic approach to research. This research was conducted at Sekolah Menengah Atas Negeri 3 Yogyakarta (SMAN 3 Yogyakarta) -- 3 Yogyakarta State Senior High School. The school understudy is one of the best schools in Indonesia. It is also one of the International Standard Pilot-Project School members in Indonesia since the academic year of 2006- 2007. The researchers made pre-observation two times in July 2014. The collection of data was done from April 2015 and it took one month. The analysis and conclusions were drawn in May 2015. In this study, the informants were composed of 43 students studying in the eleventh grade of SMAN 3 Yogyakarta. They were chosen using a purposive sampling technique from a total number of eight classes of grade eleven because the rest of the classes had different English teaching programs. In addition to the students, their respective two English teachers were also among the respondents of this research. In this study, data were collected using a variety of instruments depending upon what kind of data to be collected. It includes observation, questionnaires, interviews and documentations. As this evaluation is a qualitative study with a quantitative support, the instruments are divided into two parts. There are a qualitative part and a quantitative one. To get quantitative data, questionnaires were handed to teachers and students. The qualitative data were collected through interview which was administered to teachers. Documentation technique was used to get both qualitative and quantitative data. In analyzing the data, the three key stage methods of analyzing qualitative data Research and Evaluation in Education 121 - Volume 1, Number 2, December 2015 stated by Miles and Huberman (1994, pp.10- 12) were followed. Those stages are data reduction, data display, and also conclusion drawing. First of all, data were reduced just to make them simpler to analyze. After that, the researchers summarized the collected data after reduction, sort them without forgetting to discard irrelevant data before drawing conclusions, so that the data will be easily understood. At the end, to make general conclusions, the findings were descriptively analyzed to check if they have responded to the students’ needs and if the course has effectively reached the program’s objectives. With regard to data from surveys, i.e., some quantitative data from students’ survey, a descriptive statistics was used, based on the modified Likert scale method proposed by Mardapi (2008, p.123) as presented in Table 1. After analyzing the data quantitatively, the results were interpreted qualitatively in order to make them easier to understand. Some of them were compared to the teachers’ responses before drawing conclusions. Table 1.Students’ Perceptions for Teaching Materials, Classroom Conditions, and Teaching Methods Score X Categories X ≥ M + 1. SD Strongly agree/Always M ≤ X 50. Evaluation Criteria of the Results In this section, each question was analyzed independently and percentages were used to measure the effectiveness. To that end, each item was regarded as effective if the sum of the options ‘a’ and ‘b’ was bigger than 50%, for instance, Ʃ (a + b) >50. Findings and Discussions The findings are presented and discuss- ed referring to the four components of CIPP model. In the context component, it is found that teachers know the needs and problems of their students. It is found as a big and important point to appreciate because teachers are taken as parents. They replace the role of students’ parents at school. Thus, as good parents know when their children are hungry or not, what they like or not, teachers of SMAN 3 Yogyakarta have been found in the category of good parents. They know their students’ needs and problems. In other words, they care about their students. Second, it is found that the English Program objectives encompass most of the students’ needs and problems. It is justified by the fact that: (a) The most and main students’ needs are covered up to 66.67% of the students’ needs; (b) The most and the main students’ problems are covered up to 75% of the students’ barriers. Based on those findings, it can be concluded that the teaching English program objectives are well elaborated as they include most of the students’ needs and problems. It is one of good characteristics a teaching pro- gram should have. A good teaching program is the one whose objectives are shaped based on the recipients’ capacity, needs and problems. In short, we can conclude that the program objectives covered effectively the students’ barriers. The classroom environment is found to be conducive to the teaching/learning at the school understudy. It means that the class- room conditions fulfill all conditions of a good setting for studying. In other words, the classroom conditions are favorable in helping students to follow their learning well, by helping them to meet their needs, and finding solution to the problems they are facing while studying. It coincides with Wei (2011, p.90) in his study where the findings about relation- ship between students’ perceptions of class- room environment and their motivation in learning English language indicate that most of the students perceived their classroom as having affiliation and they were extrinsically motivated. Considering that, teachers have a big responsibility to create thrilling classroom settings for their students so that the goals of teaching activity can be achieved. Moreover, Suleman, Aslam, and Hussain (2014, p.71) find out that classroom favorable environment has a significant positive effect on the academic achievement scores of secondary school students. It can be perceived that environment plays a big role in Research and Evaluation in Education 123 - Volume 1, Number 2, December 2015 teaching. Classroom settings have to be taken into consideration by teachers while planning their lessons. Otherwise, it will be hard to achieve the teaching goals. The input component shows that both teachers are not only qualified in English teaching but also experienced with a mean of 13 years in teaching English. The first one is a ten-years-experienced teacher in teaching English and she is a master of English teach- ing. The other one is a bachelor of English teaching and has been teaching for 16 years. The combination of the two characteristics is very important in teaching domain. Those characteristics are the other factors which can be the basis of students’ success in English. At secondary level, the percentage of student scoring advanced and proficient increases as the increasing of number of years of teaching experience (Dial, 2008, p. 107). Secondly, it is found that both English teachers of SMAN 3 Yogyakarta who have followed the teaching training related to Curriculum 2013 and the teachers who are not satisfied with the training they followed in the allotted time agree that teaching training was insufficient. Thus, they were claiming for extra teaching training. Generally, in any domain and in teach- ing domain in particular, training is one of the most important key factors to take into consideration if a person wants to succeed. With training, one gets knowledge, confidence and likewise will be familiar with the material to be used. Thus, the stakeholders in edu- cation should have thought twice about teaching training before they implement English teach-ing program. If not, it can end up in smoke. Third, it is also discovered that both teachers have followed some ongoing training seminars, such as some collaborative seminars with other schools or internal seminars related to Curriculum 2013. It is an excellent strategy to use in strengthening their skills and knowledge. What happened to SMAN 3 Yogyakarta teachers also happened in Pakistan where the findings of a study about teachers’ training showed that the teachers claimed that the training program should be longer (long term training,) because it was proven that the training was effective as a useful forum to make them become effective English teachers (Wati, 2011). In the same perspectives, Farooq and Aslam (2011, p.30) embrace Wati saying that training gushes ability of working in any sort of employee, even in non- professional and new employees; it pushes up the abilities of professionals to a higher stage from where they currently stand. However, those teaching trainings have to be well organized if the government wants the training to be successful. If not, the outcome of the teaching training will be like what happened in Pakistan where many train- ing programs for English subject teachers were organized at different levels so that teachers would be equipped with necessary latest knowledge and skills to make them good English teachers. However, the results of training did not match with the desired objectives. From this condition, it is necessary for the government not only to organize training but also to give more time and devotion in order to be successful. Moreover, it is found that not all students have their personal textbooks. A number of 32 students or 74.42% of the total students have each their personal English book. It means that there are 11 students or 25.58% of the students who have no personal English textbooks. It implies that it is not easy for those who do not have their own books in case they want to revise or prepare in advance for the next lesson or simply in case they have a homework that has to be found in the book. Each student should have their own English textbooks so that they can use them whenever they need it without losing time to go to borrow the book from their friends, who probably also need to use their books. Apart from the students’ books, it is found that there are various teaching facilities such as other English books, teachers’ books, syllabus, computer lab, recorder, internet connection, board markers, white boards, head projectors, as well as audio visual facilities such as Radios, CD player, DVD player, and TV set. However, the teachers expressed that the teaching materials were insufficient. Research and Evaluation in Education The effectiveness of English teaching program... - 124 Alfred Irambona & Kumaidi Teaching materials are taken as the light in the darkness. They support a lot the teach- ing activities as they are the ones that teachers refer to in preparing and in teaching activities. They are taken as the guidelines of teaching activities. Without them, the teaching activiti- es will not get any supports, or, otherwhise, be miss-directed. In the same component, it is found that teachers at SMAN 3 Yogyakarta design the lessons by themselves. They admitted that they sometimes collaborate with their fellows, ask some expert in English teaching, or just read some articles in order to have strong and successful lesson plans. Another finding is that the elaborations of lesson plans are based on the program objectives, the needs and problems of the students. Lesson plans are one of the vital factors in teaching because they trace paths which lead to the connection between what is taught and what should be taught. In other words, lesson plans will give a connection between the objectives of the program and the content of the lesson being taught. This paragraph is concluded with a note of agreement with the saying which is stated as ‘failing to prepare is preparing to fail’. As with the process component, the findings show that the teaching/learning materials are in an ‘effective’ category. It means that they help students to meet their needs and overcome their barriers while studying English language. However, it is found that there are some teaching/learning materials which are overused while there are others which are less used. Through video and projectors which are used in up to 86.05% of the teaching activities, it can be concluded that most of the students’ needs and barriers should be re- solved there as teachers admitted that they showed a variety of videos. For example, most of the needs and problems related to different language skills may be easily resolved through movies. They can practice listening, writing a summary of what they saw in the movies, reading, they can gain lots of vocabulary items and expressions, and they can even speak. Thus, through videos or what was projected, students’ needs and barriers can be resolved. It is also found that students produce other things such as novels, a lot of posters, autobiography booklets, and had made a lot of handcrafts based on their interest-based learning. With such activities, students will produce something they like and they are interested in. It means that students at the school understudy explore all the potentiality they have in various domains so that they can meet their needs. By practicing, they face their problems and solve them in practicing. Discussing the findings concerning the teaching methods, it is found that teaching methods are effective in the point of view of both teachers and students. It means that the teaching methods are leading students to the fulfillment of their needs and in the resolution of their problems. It may have resulted in the use of various teaching activities by the teachers. It is found that communicative, scientific approaches, project-based learning, and interest-based learning are used at that school. As they are experienced teachers, they know when a given teaching methods should be applied and when it needs to be changed. For example, the way of mixing methods from old curriculum (curriculum 2006) and the new one (curriculum 2013) is a great thing to appreciate because they know which methods are effective and which ones are ineffective. The English teachers of SMAN 3 Yogyakarta look for many ways to help their students to overcome their problems and meet their needs. It is reflected in the fact that they organize discussion groups, excursions in English speaking places, and also organize students exchange programs with native English speaking students inside and outside of the country. In all those activities, the students will get many occasions to improve their English. If all teachers helped their students in such a way, all Indonesian students would be able to improve their English skills very fast. The government should support teachers even schools with such a glorious initiative of devotion. Research and Evaluation in Education 125 - Volume 1, Number 2, December 2015 As long as assessments are concerned, it is found that assessments are helping teachers to find solutions of their students’ problems and needs. Through assessments, teachers know where their students are strong and where they are weak. They get a database of information and, likewise, they know where to put emphasis on. Thus, they will know what to prepare for the next classes. It is also found that feedbacks are done in classes. With feedbacks, students will also know about their weaknesses and strengths. In such a way, students will know what to focus while studying. Regarding the last component of CIPP model, it is found that at the end of the semester, most of the students’ needs are fulfilled. It is found that a number of 31 students which stands for 72.09% of the students express that their needs were reach- ed. It means that at the end of the first semester, the students’ needs are in an ‘effective’ category. The data got from both teachers also show that most of the students’ needs are fulfilled. The comparison of the two sides shows that there is convergent point of view from both sides. From the findings, based on the effectiveness criteria, it can be concluded that the English teaching program is effectively responding to the students’ needs. In the same way, most of the students’ barriers are solved. It means that the teaching program is in an effective category as the majority of the students that is 74.42% of the students accepted that most of their barriers are revolved. The information got from both teachers shows that most of the students’ problems are solved. From the findings about the two points, it is clear that teachers are focusing on one side, i.e. on students’ barriers. It is better to make a balance between the two sides so that both students’ barriers and needs should be dealt successfully. It may also have resulted in the fact that teachers are not balancing students’ inferred needs and expressed ones. According to Arends and Kilcher (2010, p.91), determining and responding to students’ needs is one of the most complex and difficult tasks faced by teachers. From this fact, it can be deduced that teachers should have focus in areas where many students’ needs are stressed and try to weigh students’ inferred needs and the ones already stated in the curriculum. Regarding to students and teachers’ perceptions about students’ grades, it is found that most of the students that is 76.74% of the students or 33 students said that they are satisfied with their grade at the end of the semester. It is also found that all students passed the subject with a class mean of 83. From the findings, it is clear that teachers did their best in bringing their students reach and pass the national passing minimal score, which is, 75. One of the ultimate goals of education is to have all students reaching the national standard. Thus, the comparison of the find- ings can lead to the conclusion that the English teaching program was effective in that part of the teaching/learning activities. Regarding the students’ and teachers’ perceptions on the teaching/learning materi- als, the teaching/learning materials are not judged enough as the majority of the students said that the teaching/learning materials are not enough. Teachers also proved that the available teaching/learning materials are insufficient. Based on the findings, it is concluded that the teaching materials are not effectively helping students to overcome their problems and meet their needs. It is understandable because since the input component, the teach- ers had complained saying that the teaching materials are enough only for the basic purposes. Thus, the teaching methods are found not effective. The government should avail sufficient and appropriate teaching/ learning materials before the implementation of a new program to increase the success in teaching. Last but not least, the English teaching program is perceived as effective by the majority of students that is equal to 60.47%. It means that teachers taught and evaluated the program objectives. Moreover, it is reflected in the fact that all students pass the English course with good grades. Research and Evaluation in Education The effectiveness of English teaching program... - 126 Alfred Irambona & Kumaidi Conclusions and Recommendations Based on the findings from the context component, it can be concluded that English program objectives set in Curriculum 2013 presented in the Table 4 are found as effective as they cover most of the students’ needs and barriers. They are found rich as they en- compass all the language components. On one hand, it is found that the main students’ needs are covered up to 66.67%. On the other hand, it is discovered that students’ barriers are covered up to 75%. Concerning the classroom conditions, it is found that the school understudy was built in a noisy environment. However, the mean score of the analysis was 16.6, and it is includ- ed in the ‘agree’ category. It simply means that the classroom condition is conducive to meet students’ needs and overcome their barriers. From the input component, the data analysis and discussions show that both SMAN 3 Yogyakarta English teachers are qualified and experienced in English teaching. The mean of experience is 13 years in teach- ing English. However, the teaching training is found not enough. Teachers mentioned that they did not get a lot from that teaching training as long as the time allotted to that training was not enough. They claimed the need of extra training related to the 2013 curriculum. As the students’ books are concerned, the findings show that not all students have their personal students’ English book. It is found that 74.42% of the students, or 32 students from a total number of 43 students have personal students’ book. Regarding the course designs, teachers of the eleventh grade of SMAN 3 Yogyakarta admitted that they created their own course designs. They also said that they collaborated with other English teachers in sharing what they know and/or get some references in case there was something they were not sure of. In elaborating what to teach, it is found that the objectives of the program and students’ needs and barriers are at the core of the course designs. In the process component, it is found that the available teaching/learning materials are capable in helping students to fulfill their needs and overcome their problems at SMAN 3 Yogyakarta. However, there are some teaching/learning materials which are oftenly used and others which are less used. The most used are videos and projectors, while the less used are magazines and board games. The teaching methods do not only help students to meet their needs but also to solve their problems. To that end, a variety of teaching methods are in regular use such as communicative approach, scientific approach, and project-based learning. Teachers are not relying only on the teaching methods already set in the Curriculum 2013; they are also using their hidden curriculum and collaborating with other English teachers. Moreover, it is also concluded that teachers really know students’ needs and barriers. A variety of teaching activities are practiced in classes. However, some dis- crepancies are noticed, that is, the way those teaching activities are practiced in classes and the way they are expected by the students. Taking students’ needs for instance, it is found through some teaching activities that there are some students’ needs which are more practiced and others which are less practiced than expected by the students. It is the same case for the students’ barriers. In short, teachers are not totally consistent in focusing in sides or areas where students have more problems or needs. Regarding the perceptions of English teachers of the eleventh grade of SMAN 3 Yogyakarta concerning the assessments, it is concluded that assessments are very helpful. Teachers are choosing tests based on students’ needs and barriers. Feedbacks are also done just to show students where they have made mistakes and are correct, and encourage them without forgetting to compliment those who perform/do better. They are found as being effective as they are helping teachers to resolve students’ problems and help students to meet their needs. The data analysis and discussions about product component lead to the following conclusions. At the end of the first semester, the majority of students’ needs are resolved. It is the case of 31 students out of 43 students or 72.09% of the total students whose needs Research and Evaluation in Education 127 - Volume 1, Number 2, December 2015 are met. It means that the students’ needs are in an ‘effective’ category. In the same way, the students’ barriers are found to be effective as the majority students’ barriers are resolved. A number of 32 students or 74.42% of the students are in the ‘effective’ category as their barriers are solved. Concerning students’ marks, teachers admitted that all students pass the English course with very good marks. The average of the English class success is 83. A number of 33 students or 76.74% of the students are in ‘effective’ category as they mentioned that they are satisfied with their English marks. Concerning the teaching/learning materials, they are found to be insufficient. They cannot totally help in finding students’ solutions to their problems and in helping students to meet their needs. Since the begin- ning, teachers mentioned that the teaching/ learning materials are not enough. The data analysis and discussions made from the perceptions of students and teachers regarding the English teaching program lead to the conclusion that the English teaching program is good. Both teachers said that the program is effective and the majority of the students (60.47%) said that they are satisfied with the English teaching program. Therefore, from the gained data, it can be concluded that the English program objectives, classroom condition, process component, teaching methods, and the product component are effective. However, the teaching/learning materials used are not effective enough to help students achieve the best results. Recommendations To the government: (a) The govern- ment should organize more training sessions for English teachers regarding Curriculum 2013 so that teachers can be familiar with the materials to be taught; (b) the government should avail more English students’ books so that each student gets their own English textbook; (c) the government should avail new teaching/learning materials related to the implementation of Curriculum 2013. To English teachers at SMAN 3 Yogyakarta: Teachers should be more consistent in their teaching activities by balancing students’ needs and barriers with the program objectives so that all of the students’ needs and barriers can be covered. For further researchers: (a) Because this case study evaluates only the first semester, further researchers should make an evaluation of the whole academic year or the whole program; (b) Further researchers should also conduct follow-ups and periodical evaluations to keep the English program updated. Bibliography Arends, R.I. & Kilcher, A. (2010). Teaching for student learning: Becoming an accomplished teacher. New York, NY: Routledge. Cambridge Advanced Learner’s Dictionary. (2000). Cambridge Advanced Learner’s Dictionary (10 th ed.). London: Cambridge Press. Chris, K. (2009). Effective teaching in schools: Theory and practice (3 rd ed.). Cheltenham: Thornes Ltd. Farooq, M. & Aslam, K.M. (2011). Impact of training and feedback on employee performance. Far East Journal of Psychology and Business, 5(1), 23-33. Fernandez, H.J.X. (1984). Evaluation of educational programs. Jakarta: National Educational Planning, Evaluation and Curriculum Development. Fterniati, A. & Spinthourakis, J.A. (2006) National curriculum reform and new elementary school language arts textbooks in Greece. International Journal of Learning, 13(4), 1-11. Retrieved from http://www.elemedu.upatras.gr/english /images/afterniati/NationalCurriculum ReformandNewElementarySchoolLang uageArtsTextbooksinGreece.pdf IDEA Center (2002). Some thoughts on selecting IDEA objectives. Retrieved from https://txwes.edu/media/twu/content- assets/images/faculty-and-staff/idea- survery-information/SOME- THOUGHTS-ON-SELECTING- IDEA-OBJECTIVES.pdf http://www.elemedu.upatras.gr/english/images/afterniati/NationalCurriculumReformandNewElementarySchoolLanguageArtsTextbooksinGreece.pdf http://www.elemedu.upatras.gr/english/images/afterniati/NationalCurriculumReformandNewElementarySchoolLanguageArtsTextbooksinGreece.pdf http://www.elemedu.upatras.gr/english/images/afterniati/NationalCurriculumReformandNewElementarySchoolLanguageArtsTextbooksinGreece.pdf http://www.elemedu.upatras.gr/english/images/afterniati/NationalCurriculumReformandNewElementarySchoolLanguageArtsTextbooksinGreece.pdf Research and Evaluation in Education The effectiveness of English teaching program... - 128 Alfred Irambona & Kumaidi Kaufman, R. & Thomas, S. (1980). Evaluation without fear. New York, NY: New Viewpoints. Lynch, B.K. (1996). Language program evaluation: Theory and practice. Cambridge: Cambridge University Press. Mardapi, D. (2008). Teknik penyusunan instrument tesdan non tes [Test and non- test instrument arrangement technique]. Yogyakarta: Mitracendekia Press. McDavid, J.C. & Hawthorn, L.R.L. (2006). Program evaluation and performance measure- ment: An introduction to Practice. Thousand Oaks, CA: SAGE Publications, Inc. Miles, M.B. & Huberman, A.M. (1994). Qualitative data analysis: An expanded sourcebook (2 nd ed.). Thousand Oak, CA: SAGE Publications. Moore, K.D. (2009). Effective instructional strategies: From theory to practice (2 nd ed.). Thousand Oaks, California: SAGE Publications. Nuh, M. (2013). The future of this nation depends on the new curriculum. The Jakarta Post. Retrieved From: http://www.thejakartapost.com/news/ 2013/02/19/future-indonesia-depends- new-curriculum-minister.html. Sanders, J.R. & Sullins C.D. (2006). Evaluating school programs: An educator’s guide (3 rd ed.). Thousand Oaks, California: Corwin Press, SAGE Publications Company. Stufflebeam, D.L. & Shinkfield, A.J. (1985). Systematic evaluation. Boston: Kluwe Niljhoff Publishing. Suleman, Q., Aslam H.D. & Hussain I. (2014). Effects of classroom physical environment on the academic achieve- ment scores of secondary school students in kohat division, Pakistan. International Journal of Learning & Development, 4(1), 71-82. Wati, H. (2011). The effectiveness of Indonesian English teachers training programs in improving confidence and motivation. International Journal of Instruction, 4(1), 79-104. Wei, L.S. (2011). Relationship between students’ perceptions of classroom environment and their motivation in learning English language. International Journal of Humanities and Social Science, 1(21), 240-250. Zhang, G., Zeller, N., Griffith, R., Metcalf, D., Williams, J., Shea, C. & Misulis, K. (2011). Using the context, input, process, and product evaluation model (CIPP) as a comprehensive framework to guide the planning, implementation, and assessment of service-learning programs. Journal of Higher Education Outreach and Engagement, 15(4), 57-83. http://www.thejakartapost.com/news/2013/02/19/future-indonesia-depends-new-curriculum-minister.html http://www.thejakartapost.com/news/2013/02/19/future-indonesia-depends-new-curriculum-minister.html http://www.thejakartapost.com/news/2013/02/19/future-indonesia-depends-new-curriculum-minister.html