Research and Evaluation in Education e-ISSN: 2460-6995 Research and Evaluation in Education Volume 1, Number 2, December 2015 (pages 146-157) Available online at: http://journal.uny.ac.id/index.php/reid EVALUATION OF THE BRIDGING COURSE OFFERED AT A UNIVERSITY TO FOREIGN STUDENTS: BATCHES OF 2012 AND 2013 1) Beatriz Eugenia Orantes Perez; 2) Djemari Mardapi 1) State University of Chiapas, Mexico; 2) Yogyakarta State University, Indonesia 1) be.orantes@gmail.com; 2) djemarimardapi@gmail.com Abstract This evaluation was a case study of the bridging language program offered at a state university to its Developing Countries Partnership (DCP) scholarship awardee students. It evaluated the batches of 2012 and 2013. The focus of this evaluation was to evaluate the strengths and weaknesses of the bridging program. This was a summative evaluation that used Context, Input, Process, and Product (CIPP) method. It was a pragmatic parallel mix-method design research. The data analysis techniques employed in this study were descriptive qualitative and quantitative data. Fifteen students, six teachers, 10 tutors, and three administrators were involved in this study. In each component of the evaluation, strengths and weaknesses were found. The main strengths are: Clear cultural objectives, experience teachers, tutorials help students a lot. The main weaknesses are: There are no clear criteria of the level to be reached, tutors are lack of training and time to prepare, material and assessment need to be improved, students cannot attain an upper intermediate level of the language. Keywords: evaluation, bridging course, CIPP, DCP students mailto:be.orantes@gmail.com mailto:djemarimardapi@gmail.com Research and Evaluation in Education 147 - Volume 1, Number 2, December 2015 Introduction The main reason for a person to seek for education abroad is for a better or different education than the one offered in his/her country (Lee, 2012). Studying abroad programs are not only based in second language acquisition. They also offer inter- cultural competence, global awareness, academic discipline, and professional skills, factors that are considered when determining the success of an international studying program. Schools that offer international pro- grams are aware that international students need to pass through a process of adaptation. In this process, there are some occurring factors like transition and cultural shock. That is why offering bridging courses before inter- national students start their official courses has become very popular and effective in international schools. However there are many factors to take into account in designing and providing a bridging course, such as: students’ back- ground, objectives, methodology, assessment, selected materials, resources, teachers, and language of instruction as well as supports inside and outside the class. Whether the bridging course is effective or not, it will definitely impact the grades of international students in their following studies at the hosting school. Countries that speak a national language that is not a worldwide spoken language like English or French aim nowadays to offer scholarships that also include a bridging course to learn the language and culture of the country as a part of the main plan of making their language more spoken worldwide. More and more schools now opt to offer Bridging Courses (BC) to international students. These courses not only offer to level up students to the language level required, but also offer additional knowledge about the culture as the adaptation to the new environment. Learning a language cannot be separated from learning the culture. It is important that the language program offered also provides with opportunities to observe and understand the culture in a critical and positive way. Language in this context should be seen as a prerogative which paves the way for cultural adjustment. In other words, when foreign students want to master the language, they have to be more receptive to aspects of the new environment to cope better with their studies and find it easier to adjust in society. Studying abroad is a fast-growing phenomenon. Some of the main reasons are the desire of travel, or because of political changes, economic need, or cultural interact- ion. Traveling is part of the lives of many young people of university age, especially from developed countries. Nonetheless, many students from all over the world who travel are already between their 30s and 40s. It is implicit that studying abroad will lead to increased cultural capital and knowledge for the individual, improved international relations, and an extra dimension to the educational experience. Nevertheless, studying and living in a foreign country implies a transition where students need to adapt. Adaptation normally goes hand in hand with cultural shock. International students face these factors as well as dealing with studies, so that transition becomes one of the vital characteristics that international education providers need to take into account. Moving to a new country will require a process of transition, and it will lead the new comer to adapt to the new country. Adaptation is then the main goal, since the more adapted an international student is, the easier will be for him/her to focus only in his/her studies. Transition is a natural part of life, and can be experienced when moving from school to school, from house to house, from job to job. For international students, this transition means many things: new country, new school, new living situation, new language, new culture, new food, new people, and many more. Therefore, support in this stage is very important. Indonesia is a country that offers a scholarship called Beasiswa Kemitraan Negara Berkembang (KNB) or Developing Countries Partnership (DCP) scholarship to other Research and Evaluation in Education Evaluation of the bridging course offered at a university... - 148 Beatriz Eugenia Orantes Perez & Djemari Mardapi developing countries’ students. Universitas Negeri Yogyakarta (UNY) -- Yogyakarta State University (YSU) is one of the universities in Indonesia which has become the host of the DCP program since 2006. YSU is located in Yogyakarta city, a special region of Indonesia, where Javanese is the first language for most of the local people. In Yogyakarta, Indonesian language is the official language mainly used at school and in formal activities, but the society is ruled mainly by the local Javanese language. The Developing Countries Partner- ship program demands different periods of study: A bridging course where Indonesian language and culture is studied for 8 months; a Bachelor or Master’s Preparatory Program for 4 months (matriculation can be included); and either a Bachelor’s Program of 4 years (8 semesters) or a Master’s Program of 24 months (4 semesters). The bridging course (BC) in YSU also involves Indonesian students as tutors and companions for foreign students if they have difficulties. At the end of each semester, an internal evaluation of teachers and tutors is conducted. However, they have not had an external evaluation since they have just started offering the program. Thus, the main objective of this case study research is to conduct an external evaluation of the bridging course offered at YSU to describe the strengths and weaknesses of the program. This evaluation is summative and covers a period of two batches of 2012 and 2013 of DCP awardee students at YSU. DCP students who arrive in Indonesia for the first time have little or null knowledge of the language, and they have 8 months to learn and perfect their Indonesian language to a level that allows those selected students to be competent inside their Bachelor or Master’s degree’s classes, and also inside the society. However, the course has not been evaluated yet. Therefore, it is the importance of this evaluation. The needs referred in this study are the needs of the university regarding international students (requirements) and the students’ needs inside a classroom (expectations); however, we cannot truly separate students’ personal needs with classroom needs. The job of a teacher while planning a lesson is to understand student’s need and expectations about the learning process they are under. Moore (2009, p.337) explains that teachers need to take into account students’ needs and provide students with different assignments that fulfil those needs. Thus, getting to know students’ background includes not only getting to know what their learning needs are, but also there might be some needs that might affect their learning process, for instances: lack of instruments to do a homework, cultural shock, or problems at home. This research also evaluates the university’s language requirements and the students’ needs, along with the objectives of the course regarding to the language and culture and whether they are reached; as well as the resources which are used, the training and experience of the teachers as well as the methodology and activities which are chosen, the language level reached by students as well as their cultural awareness. In 1993, the government of the Republic of Indonesia started offering postgraduate (master degree) scholarships to students from other countries. In 2009, it was recorded that more than 400 students from more than 40 countries had been awarded this scholarship. The program is now named Beasiswa Kemitraan Negara Berkembang (KNB) or Developing Countries Partnership (DPC) scholarship. Scholarship is offered by the government of the Republic of Indonesia to developing country students through this program (Directorate General of Higher Education, 2014). There are many approaches that can be employed to conduct an evaluation of a program. Fitzpatrick, Sanders and Blaine (2010, p.114) argue that ‘The diversity of evaluation approaches has risen from the varied backgrounds, experiences, and world- views of their authors, which have resulted in diverse philosophical orientations, and methodological and practical preferences.’ One of the most commonly used models is the CIPP proposed by Stufflebeam in 1970. The meaning of the acronym CIPP is: Context, Input, Process, and Product Research and Evaluation in Education 149 - Volume 1, Number 2, December 2015 evaluation. CIPP is an approach that matches many educational evaluation’s objectives. With CIPP, Stufflebeam has pointed out the use of multiple methods, both qualitative and quantitative as far as it fulfils the needs of the evaluation (quoted in Fitzpatrick, Sanders, and Blaine, 2010, p.176). The four types of evaluations are described by Stufflebeam, Madaus, and Kellaghan (2002, p. 279) as follows: the context evaluation assesses the needs, problems, assets and opportunities; the input assesses the strategy and the associated work plan and budget for carrying out the effort; the process checks the plan’s implementation plus documentation of the process, including changes in the plan as well as key omissions and/or poor execution of certain procedures; and the product accesses the extent to which the evaluation met the needs all the rightful beneficiaries, and it gives feedback about achievements, and helps to take decisions about the program, whether it is worth using, continuing, repeating, and/or changing. When the CIPP model is taken as a model to evaluate a program that has already finished, it is said that it is a summative evaluation, and the main purpose is to sum up the program’s merit, worth, probity, and significance (Fitzpatrick, Sanders, & Blaine, 2010, p. 175). When the evaluation is summative, according to Stufflebeam, the four different evaluations have role as follows: (1) Context: It is useful for judging the goals already established and for helping the audience to assess the significance of the effort in meeting the beneficiaries’ needs. Thus, it compares the goals and priorities with the assessed needs, problems, assets, and opportunities; (2) Input: It compares the program’s strategies, design and budget to those of critical competitors and the targeted needs of beneficiaries; (3) Process: It provides a full description of the actual process and cost, plus comparison of the designed and actual processes and costs; (4) Product: Finally it compares the outcomes and side effects to targeted needs and as feasible, to results of competitive programs; interpret- ation of results against the effort’s assessed context, inputs, and processes. Research Objectives The primary purpose of this evaluation is to examine the strengths and weaknesses of the program. It is necessary to evaluate the program relevance, program management and delivery, without forgetting to conduct an assessment of the impact from the DCP bridging program. It assesses the extent to which DCP participants improved their language abilities and acquired knowledge of Indonesian culture, the instruments and methods used to deliver language instruction, as well as the assessment of instruments, and the barriers to access the program. It also examines if the social and students’ needs are fulfilled during the program to be able to participate effectively in the lesson in Bachelor or Master’s degree levels. Thus, the research objectives are as follows: (1) To know what YSU require- ments to enter a master degree are and what the students’ background is; (2) to know what the participants’ needs are; (3) to understand the learning goals/objectives of the course; (4) to evaluate if the program was designed to fulfil students and university’s demands; (5) to determine what and how good the resources are; (6) to evaluate if the planning of the lesson is based on the objectives of the course and the needs of students; (7) to know the teacher’s previous experience and training in teaching Indonesian language to foreign students; (8) to evaluate the methodologies used; (9) to evaluate if the student’s language level and cultural awareness reached after following two semesters of bridging course at YSU is the optimal result; (10) to evaluate if the needs and objectives are fulfilled at the end of the course; and (11) to determine the strengths and weaknesses of the course. Research Methods This evaluation is a case study. To be able to evaluate this program, a summative evaluation was conducted by using CIPP method. This evaluation collected the data quantitatively and qualitatively. This research was a pragmatic parallel mix-method design research, since both quantitative and qualitative data were collected at the same Research and Evaluation in Education Evaluation of the bridging course offered at a university... - 150 Beatriz Eugenia Orantes Perez & Djemari Mardapi time or at a short period of time (Mertens, 2010, p.298). The analysis technique was descriptive for the qualitative data and it was descriptively statistical for the quantitative data. Triangulation was used to compare the results of the analysis of the data from the questionnaires, interviews, outcomes from the course, and relevant bibliography. The research was conducted in Yogyakarta State University (YSU), for a period of one year. It started on June 2014 and finished on June 2015. The subjects of the evaluation were: six students from 2012 batch and nine students from 2013 batch, six teachers, ten tutors and three administrative staffs. Several lines of inquiry were used to evaluate the program: document review; literature review; administrative data analysis; questionnaires and interviews to DCP students, teachers and administrative staffs. The validity of the instruments was focused on the content and format of the instruments. In order to confirm that the instruments measured what they were supposed to measure, a pilot of the check list, the questionnaires and interview paper were conducted. In addition, three experts were appointed to validate credibility of the instrument. All of the instruments were improved. The research data were divided in three main sectors: Analyzing (from the researchers’ point of view), interviews and questionnaires (from the stakeholders’ point of view) and results (the outcomes at the end). First, since this was a mix-method research, the data were divided and analyzed in quantitative and qualitative method. Second, the procedure of analyzing the data was also different accord- ing to the type of the method used. Qualitative Data All data from the interviews were qualitative. A few questions from the questionnaires were quantitative; except for the questionnaires for administrative staffs, there were only two administrators answered the questionnaires, so that the Likert scale could not be used and a qualitative approach was employed. The qualitative questions are divided into two groups: descriptive questions for all participants and some descriptive questions for administrators. The criteria of the qualitative questions of the questionnaires for administrators were: for option ‘a’ or ‘b’, it was considered as the ‘strength’ of the pro- gram; for option ‘c’ or ‘d’, it was considered as the ‘weakness’ of the program. The data analysis strategy chosen for the descriptive qualitative data was the one presented by Hesse-Biber and Leavy (2006, quoted by Mertens, 2010, p. 424) that consists of three steps: (a) Preparing data for analysis (organizing the data); (b) data exploration phase (reading, thinking and making notes); (c) data reduction phase (selecting the relevant data and assigning a label). Quantitative Data The quantitative data were analyzed in different methods: Likert, percentage, and mean. Each method is explained as follows. The Percentage Method Percentage was used as a method to analyze some of the quantitative data. There were two types of questions. One type of questions gave the stakeholders opportunity to choose only one option. The frequency of occurrence of these questions was divided by the number of respondents who answered it: two administrators, six teachers, ten tutors, and fifteen students. The formula goes as follows: Percentage: Frequency Number of stakeholder per group The other type of questions was multiple answers. Stakeholder could choose or give as many answers as they felt it was necessary. The frequencies of these questions were sum and then each frequency was divided by the total number of the questions. The formula is as follows: Percentage: Frequency Total number of the questions Modified Likert Scale The modified Likert scale was based on the categories of behaviours and interests Research and Evaluation in Education 151 - Volume 1, Number 2, December 2015 proposed by Mardapi (2008, p.123), where the categories of behaviours and interests are ranked by using the following formula: Table 1. Categories of Behaviours and Interests Score X Categories X ≥ M + 1. SD Very High M ≤ X < M + 1. SD High M - 1. SD ≤ X < M Low X < M - 1. SD Very Low For this case study, each question was analyzed individually and the criteria used were shown in Table 2. Table 2. Category of the Likert Scale Criteria Category X ≥ 3 Strong 2.5≤ X <3 Strong 2 ≤ X< 2.5 Weak X < 2 Strong Weak The criteria were made taking the following elements into account: Number of Question : 1 Maximum : 4 Minimum : 1. Sd : 0.5 M : 2.5 The criteria for all the questions that were analyzed using the modified Likert scale were the following: ‘a’ equals to four points, ‘b’ equals to three points, ‘c’ equals to two points, and ‘d’ equals to one point. Then each answer given by the stakeholders was replaced by the value and the mean was obtained to be able to know under which category on Table 1 and 2 the question belongs to. Mean There were two types of mean used: a simple mean and a mean for ranking. The simple mean to get the average years of teachers’ and tutors’ teaching experience was by adding all the numbers and dividing the result between the numbers of participants. The second type was when using ranking. According to the Statistical Services Center (2001, p.6), ‘the ordered categories can be reduced by accepting a degree of arbitrari- ness. Then, give scores to the categories and produce an average score.’ For this case study, the questions where rankings were asked followed the criteria of: ‘1’ equals to seven, ‘2’ equals to six, ‘3’ equals to five, ‘4’ equals to four, ‘5’ equals to three, ‘6’ equals to two, and ‘7’ equals to one. Then, the frequency of each category, in a question, was multiplied by the criteria and then sum, and divided by the sum of all frequencies (N). The mean of each question then was taken into account. If the mean was μ<4, the option of the question was consider- ed as the ‘weakness’ of the program. If it was μ≥4, then it is considered as belong to the category of the ‘strength’ of the program. μ = (f1*7+ f2*6+ f3*5+ f4*4+ f5*3+ f6*2+ f7*1) N Findings and Discussions Context For any foreign student, YSU requires an ‘upper intermediate’ level of language to enter the university. Thus, an exam is applied to measure in which level a foreign student belongs to. However, for DCP students, there is no language requirement to enter to the bachelors or masters since the program only requires students to follow an Indonesian language program, so there is no need to reach any level. Therefore, there is no clear language level required for DCP students that is administered by the administrators, and also teachers and tutors of the program have no clear level for the students’ target achieve- ment. The objectives of the program also lack to include specific academic language inside the program regarding to students’ field of study. It cannot help students to reach their first need ‘to learn the language to be competent in the class’. The syllabus did not personalize the vocabulary into different areas of study too, so that it can be considered as being lack in fulfilling one need of students. Brown (2007, p.78) explains students need to communicate in a contextualized, appropriate and meaningful way. Being able to fluently communicate in their field of study will increase students’ confidence and grades. In addition, the syllabus is too general, leaving too much room for ‘flexibility’ which is a Research and Evaluation in Education Evaluation of the bridging course offered at a university... - 152 Beatriz Eugenia Orantes Perez & Djemari Mardapi good characteristic for the experienced teach- ers, but not a good one for the inexperience tutors. Threfore, the language’s objectives and the syllabus can be considered as a weakness of the program. The strengths of the program are the part of cultural awareness where the pro- gram’s objectives are clear, bringing awareness on Yogyakarta as the main objective. Moreover, most of the objectives are related to the communication outside the school, interaction with the society, and with their daily life activities. All of the four language skills are covered, together with grammar. The two main needs of students, ‘to learn the language to be competent in class,’ and ‘to learn the language to communicate outside school’, are considered by the administrators, teachers and tutors as important needs. However, the third main need of students, ‘to adapt the course to skills and needs,’ was not so considered by the administrators, teachers and tutors as a very important need. In the class, students will show different abilities or skills. Being able to recognize, plan, and bring activities that cover as many skills as possible will help all students to learn. Input Although the materials are considered as having good quality, the materials and extra materials in general are not enough; the books lack strength since they are not used widely by the teachers and tutors. Tutors also claimed that the materials for them are insufficient. It can be considered as a ‘weakness’ of the program. However, the resources in the class are enough; it is considered as a ‘strength’ of the program. Lessons plans were not given to the administrators, and this is an important part of the assessment of students’ learning. Moreover, teachers and tutors did not always plan the lessons based on the syllabus, since the syllabus is too general. Teachers and some tutors declared that they planned their lessons based on what they felt students needed to learn. The lesson planning should be balanced between the students’ needs and the syllabus to fulfil the objectives of the program. Thus, the way the planning of the lesson has been made can be considered as the ‘strength’ of the program, but the fact that teachers and tutors do not hand the lesson planning can be considered as the ‘weakness’ of the program. Teachers have sufficient experience in teaching and teaching Indonesian Language to foreign students. Therefore, training for three days is more than enough for them. This is ‘strength’ of the program. However, tutors are less experienced and a three-day-training is not enough for them. It can be considered as the ‘weakness’ of the program. Before the students started the course, there was no information given to the teach- ers or tutors related to students’ background and level of English. Tutors were not given enough time or support to prepare their lessons. They are inexperienced, so that this lack of information affects their performance. These two last points are considered as the ‘weaknesses’ of the program. Process All of the four skills and grammar are included in the program. It is considered as the ‘strength’ of the program. However, when balancing the activities, it is noticeable that the activities are mainly focused on ‘speaking’, ‘reading’ and ‘listening’. ‘Pronunciation’ and ‘grammar’ are not so often practiced. In addition, regarding the barriers faced by students, some mentioned that they could not pronounce Indonesian words very well and that they lacked more grammar knowledge. Thus, these skills need to be balanced. Balancing the activities is necessary in any language program; otherwise, students will end up being great at some skills but lacking on others. Grammar should not be diminished, since to be able to compete in a language, it is necessary to be accurate. It can be considered as a ‘weakness’ of the program. Feedback is considered as a ‘weakness’ of the program, since it is not constantly given and this is one of the most important factors in seeking students’ improvement. Support from the tutors, on the other hand, is considered as ‘strength’ of the program, since it is constantly given. Yet, support form the Research and Evaluation in Education 153 - Volume 1, Number 2, December 2015 teachers is not constantly given, so that it can be considered as a ‘weakness’ of the program. However, students, teachers and tutors are satisfied with the way the assessment was done. Assessment also needs to be reinforced since not all the participants know when and how the assessment should have been done. In addition, there was no control from the administrators, there was no clear database of the results; there was no consistency on the way the grades were provided for the two semesters. Students did not know the level they achieved. Thus, it can be considered as the ‘weakness’ of the program. Product Administrators agreed that the results of the DCP students were ‘as expected’ in general after the two semesters at the bridging course at YSU. More than half of the students, most of the teachers, and the tutors also agreed that the students were ‘prepared’ at the end of the course. Administrators and teachers agreed in the questionnaire that the course brings students to a certain level but not to an upper intermediate level. Students also expressed that they were prepared but they faced problems. Students are being prepared as expected can be considered as the ‘strength’ of the program, but the level of Indonesian language needs required for the program needs to be clear among all stakeholders. ‘The tutorial classes’ are seen as the main the characteristics that help students to reach the level of Indonesian language. For this reason, the tutorials lessons need to be more taken care of, for example: tutors need to have a better training and more time to prepare the lessons. The tutorial, as part of the strategy to help students to reach a good level of language, can be considered as the ‘strength’ of the program. However, the way tutors are performing the lessons are still not satisfying to students. As explained by Killen (2009, p.77), the way students learn is important because their learning experiences will directly influence their motivation and their future learning strategies. Thus, the methodology used by tutors can be considered as the ‘weakness’ of the program, and this goes together with the training given. The program’s objectives are ‘clear and realistic’ and ‘taking students’ needs into account’. The objectives are realistic in terms of helping students to adapt and bring them to an intermediate level of communication. However, the program fails to take students to an upper intermediate level that, according to the administrators, is the level that any foreign student should have to be able to follow lessons at YSU masters or bachelors. Students themselves confirmed that their level of language was intermediate, and during their first semester, they faced problems like: ‘they were not able to understand the language’, ‘it was not like in the bridging course’, ‘they were not able to fulfil assignments completely in Indonesian language’, ‘they were not able to speak fluently’, and other problems. Therefore, the objectives of the program regarding cultural awareness can be considered as the ‘strength’ of the program, but the level of language that is needed to follow a masters or bachelors’ degree is considered as the ‘weakness’ of the program. Students, teachers and tutors expressed that ‘students were aware of Indonesian culture but mainly about Yogyakarta’. The main reasons are because: ‘the syllabus was designed to promote cultural awareness’ and ‘the material was designed and selected to promote cultural awareness’ and according to the students, ‘Indonesian culture is similar to their culture’. Being aware of Indonesian culture, especially Yogyakarta, is the main objective expressed by the administrators, teachers and tutors. Thus, it can be considered as the ‘strength’ of the program. Regarding adaptation, administrators, teachers, and tutors agree that ‘most of them adapted very well and a few of them are just well’; more than a half of the students agree that the bridging course helps them mainly to interact with people. In general, the students’ adaptation to society is well covered by the course. Thus, it can be considered as the strength of the bridging course. Research and Evaluation in Education Evaluation of the bridging course offered at a university... - 154 Beatriz Eugenia Orantes Perez & Djemari Mardapi On the other hand, regarding whether the bridging course help them to integrate to their masters or bachelors’ lesson, more than half of students chose the option ‘somehow, but there are some things I did not learn in the classes’. These things that students wished to know before entering their masters or bachelors’ course are: ‘culture inside the class’, ‘Javanese language’, and also ‘the learning style’. During the interviews and the recommendations given to improve the course, students mentioned that academic language is one of the things they lacked during the bridging course. For their adapt- ation to the lessons, there are still some needs that need to be covered by the course, so it can be considered as the ‘weakness’ of the program. Students expressed that they were ‘satisfied’ with the course, and ‘satisfied’ with the testing model. They also mentioned that the methodology was ‘most of the times adapted to their learning styles, but in a few times, they did not feel the teaching methodology was adequate for them’. One student mentioned in the interview that his/ her needs were not totally fulfilled regarding to the knowledge. Another student mentioned that he/she felt not prepared to face the course, since he/she had poor academic language knowledge. If we compare these results with the main students’ need that is: ‘to learn the language to be competent in class’, first, it is important to remark that the administrators agreed that the ideal level to enter a masters or bachelors’ degree is upper intermediate level. However, the course itself cannot lead a student to that level since the objectives are not clear. Therefore, students are struggling in their first semester of masters or bachelors’ degree. The main need of students is not completely fulfilled, so it can be considered as the ‘weakness’ of the program. The second need of students is: ‘to learn the language to communicate outside the school’. This need is for sure fulfilled since most of the activities are ‘speaking activities’. Students also declared that the bridging course helps them to ‘interact with people’. This can be considered as the ‘strength’ of the program. The third main need of students is ‘to adapt the course to skills and needs’. As observed in the activities presented, they were varied covering the four skills: ‘speaking’, ‘reading’, ‘writing’ and ‘listening’. However, ‘grammar’ and ‘pronunciation’ were not widely practiced, and some students, in the interviews and advices to improve the pro- gram, wrote that it is important to conduct more practice. Students evaluated their teachers in a positive way, agreeing that they appreciated their experiences and support. However, feedback, support, punctuality, and teaching methods are recommended by some students as the features that need to be improved by the teachers and tutors. Teachers’ perform- ance can be considered as the ‘strength’ of the program. For tutors, students also considered their performance as positive, since it helps them to understand the language, and the tutors were friendly and helpful. However, the teaching methods and feedback were negative aspects that the tutors need to improve. Tutors performance can be considered as the ‘weakness’ of the program since more than 50% of the students are not satisfied with the methods used by the tutors. There is no report of any evaluation which has been made to the course during the years that the bridging course has been offered. It is contradictory with the advice expressed by Garavalia, et al. (1999, p.15) that: Faculties which are interested in improving the quality of their syllabi should obtain feedback from a variety of sources including students, other faculty members and administrators, and personal reflection. Conclusion and Suggestions Conclusion In the context, the course has clear objectives about the awareness of culture that should be reached by DCP students and it promotes communication skills, as the practice of the four language skills. The stakeholders are aware of the two main needs Research and Evaluation in Education 155 - Volume 1, Number 2, December 2015 of students: ‘to be competent in class’ and ‘to communicate outside the school’. On the other hand, the level of Indonesian language that DCP students should reach is not clear, and it is not the same as requested to non- DCP foreign students. Thus, the syllabus fails to include academic language specific to the field of study of the DCP scholarship participants. From the evaluation of the input, it is found that the lesson planning is balanced between the students’ needs and the syllabus. The training provided to teachers and tutors is around 4 days; for teachers (who already have sufficient experience), the training is enough, but for tutors who have lack of experience, the training is not enough; in addition, training on how to handle students who do not speak English is not provided. There are a variety of material and resources provided by the administrators. In contrast, the materials are not enough for the teaching of the course. Lessons plans are not handled to the administrators neither by teachers nor by tutors; in addition, tutors are given not enough time to prepare the lessons. Another weakness is that teachers and tutors are not given specific information about DCP student’s background. The process’ results show that support is given to students during their adaptation. This support is constantly given by tutors but not by teachers. On the other hand, teachers’ methods are suitable to students, but tutors’ methods are not. Nevertheless, bridging course is applied to guaranty students’ adaptation with a range of activities and material, regarding to cultural understanding. Pronunciation and grammar are found not usually practiced and feedback is not constantly given. Administrators, teach- ers, tutors and students do not know when and how the assessment should have been done. The monitoring offered to teachers and tutors is constant. The last part of the evaluation is product; the results show that the program helps students to adapt and bring them to an intermediate level of communication. The tutorial classes play a key role since it supports students to improve their level of Indonesian language. In addition, students are aware of Indonesian culture, mainly the culture of Yogyakarta. In general, students evaluate their teachers in a positive way; the experience possessed by teachers is the main character- istic appreciated by students. Tutors are considered as being positive by students since they help students to understand the language; and they are friendly and helpful. Despite the good results, students do not reach an upper intermediate level of Indonesian language. In addition, there are some adaptation problems inside the class- room that are not covered by the program, such as lack of academic language, or problems with Javanese language. The main need of students is not completely fulfilled, since they do not reach a level of the language that allows them to be competent in class. In addition, teachers’ feedback, punctuality, and teaching methods, as well as the tutors’ performance do not completely satisfy students. Control from the administrators regard- ing the results database is weak, since they do not have all the results from students and the way the grades are presented in the diplomas is not consistent. At the end, students are not able to know their level of language. Suggestions Administrators of the BC KNB YSU It is necessary to establish the goals of language level and passing criteria; It is also important to include academic language specific of the area of students’ bachelors or masters degree to personalize the vocabulary by creating opportunities like reading, listen- ing, writing or presenting. Teachers and tutors need to be given more information about students’ background like their previous studies, skills (language, computer, and so on), and difficulties while learning, life style, personality and many more. Tutors should be given more training about lesson planning, improvisation, and methods. The training should include: effective feed- back and strategies to deal with students who do not speak English or the target language. Research and Evaluation in Education Evaluation of the bridging course offered at a university... - 156 Beatriz Eugenia Orantes Perez & Djemari Mardapi Another important point is that the lesson plans should be requested to teachers a week in advance, as a way to help tutors to plan better and give support if it is requested. Evaluating the materials and then revising the materials together with teachers, tutors and also students if possible to improve them is also recommended. In addition, the assess- ment system needs to be revised and controlled. If possible, a test by the administration should be done to have a control of the students’ level. Moreover, the way the diplomas present the results should be improved providing the standardized criterion and more information about the level acquired. In addition, a syllabus has to be made, taking into account that DCP students have eight months to reach an upper intermediate level of the language, where more focus on the language learning as well as academic language personalized to areas of study should be paid. It can also be possible to do follow- ups to students after their first semester in their master or bachelor programs, to know what problems they face and include this in the following bridging courses. Teachers and Tutors The teachers and tutors are expected to know students’ expectations and limitations. It is also necessary to give more support and practice in pronunciation as well as in grammar. In addition, activities that promote students to learn more about their target either in master or bachelor degree should also be included. A key point that needs to be improved is feedback. Feedback is important, and it needs to be given constantly and effectively, since students can learn from their mistakes and progresses. For teachers, it is suggested that punctuality should be considered as a characteristic that is important to students, and more constant support is required by the students. For tutors, more preparation or reading about the methods that can be implemented in class is necessary. Moreover, looking for support is also important. Future Researchers If someone is interested in conducting a research on the DCP bridging course, the following elements can be evaluated independently: the methods offered in the tutorials, the materials used in the bridging course, the problems faced by DCP students on their first semester, the culture inside a class a foreigner should know, and the academic language needed. References Brown, H.D. (2007). Teaching by principles: An interactive approach to langauge pedagody (3 rd ed.). White Plains, New York: Pearson Education. Fitzpatrick, J.L., Sanders, J.R. & Blaine, R.W. (2010). Program evaluation: Alternative approaches and practical guidelines (4 th ed.). New Jersey: Pearson Education. Garavalia, L.S., Hummel, J.H., Wiley, L.P., & Huitt, W.G. (1999). Constructing the course syllabus: Faculty and student perceptions of important syllabus components. Journal on Excellence in College Teaching, 10(1), 5-21. Killen, R. (2009). Effective teaching strategies: Lessons from research and practice (5 th ed.). South Melbourne, Vic: CENGAGE Learning. Directorate General of Higher Education. (2014). KNB program [DCP program]. Jakarta: Directorate General of Higher Education. Retrieved on July 23 rd 2014, from:http://www.knb.dikti.go.id/index. php. Lee, M. (2012). History of study abroad: Part 1 (1190 – 1900), Go Overseas. Retrieved on November 27 th 2014, from: www.gooverseas.com/go-abroad- blog/history-study-abroad-part-1. Mardapi, D. (2008). Teknik penyusunan instrumen tes dan non tes [Test and non- test instrument arrangement technique]. Yogykarta: Mitra Cendekia. Mertens, D.M. (2010). Research and evaluation in education and psychology: Integrating diversity Research and Evaluation in Education 157 - Volume 1, Number 2, December 2015 with quantitative, qualitative and mixed methods (3 rd ed.). Thousand Oaks, CA: SAGE. Moore, K.D. (2009). Effective instructional strategies: From theory to practice (2 nd ed.). Thousand Oaks, CA: SAGE. The Statistical Service Center. (2001). Approaches to the analysis of survey data. Biometrics advisory and support service to DFID. Reading, UK: The University of Reading Statistical Services Centre. Retrieved on February 22 nd 2015, from: http://www.reading.ac.uk/ssc/resource s/Docs/Approaches_to_the_analysis_o f_survey_data.pdf. Stufflebeam, D.L., Madaus, G.F., & Kellaghan, T. (2002). Evaluation models: Viewpoints in education and human services evaluation (2 nd ed.). Dordrecht: Kluwer Academic.