Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 65 Students Academic Progress Analysis at the Escola Politècnica Superior of the Universitat de les Illes Balears: Background implications Miquel Roca*, Yolanda González, Ramon Mas, Joan Rossello, Loren Carrasco, Francisco Forteza, Biel Cardona, Mercè Llabrés, Carlos Guerrero Universitat Illes Balears * Corresponding author: miquel.roca@uib.es Received: 2016-04-16; Accepted: 2016-08-18 Abstract In this work, the influence of the background of the University students is analyzed. In particular how the average mark of the students affects their academic progress. An anonymously collected data analysis is performed, Among these data are the number of European Credit Transfer and Accumulation System (ECTS) enrolled, the mark exams, average mark exams, access type, etc. Conclusions of each considered degree are presented at the end of the work. Keywords Background, Academic performance, access exams. http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 66 1. Introduction One of the main concerns of degree management teams in Spanish universities is the analysis of student academic progress, in order to find the main causes when this is not satisfactory, and implement improvement processes wherever possible. Successful student progress is of particular interest when it comes to the degrees that have been adapted to the European Higher Education Area. It is supposed that the Higher Education System has evolved to a system based on competences evaluation, a system focused on student work, a system which considers continuous evaluation processes, that is, a system where the teaching-learning process should imply better results in comparison to the educational systems that was in place before the Bologna Plan. In the case of engineering degrees, success rates are usually worse than in other degrees. In fact, these degrees are considered as difficult ones in our society, therefore the analysis presented in this paper is highly significant in this kind of degrees. In addition to that, the greater part of Engineering Schools in Spain are involved in the processes of degrees’ accreditation renewal because they have been taught their degrees for six years and an analysis of the results is now mandatory, in order to detect strengths and weaknesses and generate improvement plans wherever needed. The process of renewal accreditation (ACREDITA [1] & ACREDITA + [2]) will end with the drafting of a self-report from the university and their evaluation from a Panel of Experts appointed by the National Evaluation Agency ANECA or by the Regional Evaluation Agencies, in some cases. Due to all the above a special motivation to develop these studies, analysis and reflections as the ones proposed in this work is highlighted. Similar studies can be found in the literature [3-6] where colleagues from the EUETIB analyze these factors at their Engineering School. A comparison of results can be found here and differences and similarities between them are commented. http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 67 The paper is structured as follows. In section 2 the main details of how the study is performed and how the data is analyzed are presented. Section 3 deals with the presentation of the obtained results and a first comment about their causes. Finally in section 4 the main conclusions of the work are pointed out. 2. Performed study The main objective of this work is to study the correlation between the average mark of the students and their performance throughout their academic career. At a second phase we have studied the relation between the ECTS obtained in the first year and the ECTS obtained along the rest of the degree, as well. Marks considered during the study are calculated with the high-school average grades and score on the University specific entrance exam. The study considers all the different degrees taught at the Escola Politècnica Superior of the Universitat Illes Balears (UIB): Industrial Electronic and Automation Engineering Degree (GEEI), Informatics Engineering Degree (GEIN), Construction Degree (GEED), Agri-food and Rural Environment Engineering Degree (GEAM), Telematics Engineering Degree (GTTT) and Mathematics Degree (GMAT). The study takes into account 2625 students from years 2009-10 to 2014-15. The students’ distribution according to the different year of enrollment and degree is presented in Table I. http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 68 Table I. Students distribution (academic years and the different degrees) Year 09/10 Year 10/11 Year 11/12 Year 12/13 Year 13/14 Year 14/15 Total GEEI 126 55 71 65 69 386 GEIN 175 132 160 158 141 766 GEED 86 83 205 249 142 56 821 GEAM 57 37 54 46 41 235 GTTT 41 31 76 50 44 242 GMAT 29 26 31 41 17 31 175 The data analyzed have been obtained from the UIB Students Service and from the Information Technologies Center (CTI), through spreadsheets files in Excel format. The data are anonymous, where every student is indicated by a code, and it is not possible for us to identify the student, the anonymity of the data is guaranteed. The data process has been done using MatLab. A first translation from Excel to MatLab has been made and after the results and graphs have been obtained through MatLab programming. As an example, figure 1 shows the evolution of first year students as the ratio between ECTS overcome and ECTS registered (normalized ratio) versus the mark, as it has been defined previously. http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 69 Figure 1. ECTS passed/ECTS enrolled in the first course vs average mark for the case of GEEI students. Each point of the graph represents one student. The x-axis of the graph represents the average mark of the students. The y-axis represents the ratio between the number of ECTS credits passed in relation to the number of ECTS credits enrolled in the first year. From these distributions, correlation between data is found and conclusion extracted. 3. Results. Causes Analysis. If we take a look at the case of GEEI students, we can distinguish three different zones in figure 1. The first one corresponds to the left part of the graph (marks between 5 and 6). In this zone the percentage of ECTS overcome is very low. This zone includes the students with low marks and a low performance in the first year of study at the Engineering School. The second zone covers students with a mark of 6 to 9. In this zone students’ performance varies widely. This group includes the students with medium mark of access exam. In this set, some students are easily adapted to academic demands of their program of study while others aren’t and present some adaptation problems in the transition from high school to 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 Average mark E C T S p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) GEEI http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 70 university degrees. Finally, the third zone represents students with marks between 9 and 14. In this zone the students’ performance is very high. The observation can be made that there may be a stronger correlation in the case of students of lowest and highest marks and their performance, while in the case of students with medium mark there is no clear correlation: there are some students with high performance, others with medium performance and others with low performance, independently of their mark. Figure 2. ECTS passed/ECTS enrolled vs average mark for the case of GEEI students. Following the same approach, looking now at the case of student performance during all of their academic career (figure 2), and not only in the first year of study, a very similar result is observed. The pattern is almost identical. Finally, figure 3 reflects the success during the degree versus the success in the first year of study. A very high correlation of results can be observed in this graph. Students showing a high performance in the first course also have a high performance during all the degree and vice versa, low performance in the first course usually indicates the same trend all throughout the degree. 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 Average mark E C T S p a s s e d /e n ro ll e d ( a ll c o u rs e s ) GEEI http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 71 Figure 3. Performance all courses vs Performance at the first course in the case of GEEI students. It may seem that the high marks range of access exams is set at high mark (above 9 in the analyzed case), but it must be taken into account that this mark is over 14, where 2 points could be obtained from the specific part of the University entrance exam. In this way, for instance, a mark of 9 could be obtained from scoring 7 in the general part of the exam, as the mark of access is done by the weighted average (60% high school mark and 40% general part of the access exam) plus a value between 0 to 2 depending on the result of the specific part of the exam. In the case of GEIN degree (Figures 4, 5 and 6), it can be observed as the zone for high mark students starts at similar values as in the previous analyzed GEEI degree case. It can also be deduced that there is also a correlation between these high marks and the students’ performance. On the other hand, there does not seem to be a correlation between low entrance grade and student’s (or this zone is very narrow). There is also a high number of students with medium entrance mark who present extremely low performance throughout their studies. This could be due to a high number of students being very interested in Informatics (a very attractive field in our society) who do not know the difference between Informatics user and Informatics engineer, and consequently they have some difficulties to 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 ECTS passed/enrolled (1st course) E C T S p a s s e d /e n ro ll e d ( a ll c o u rs e s ) GEEI http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 72 progress in the university degree. These students often have little or no background in mathematics and science aspects. The behavior in the performance throughout the degree is almost the same as when looking at considering first course performance. As in the previous degree a significant correlation between performance throughout the degree and student performance in the first course can be observed, although the correlation is slightly weaker in this case. Figure 4. ECTS passed/ECTS enrolled in the first course vs average mark for the case of GEIN students. Figure 5. ECTS passed/ECTS enrolled vs average mark for the case of GEIN students. 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 Average mark E C T S p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) GEIN 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 Average mark E C T S p a s s e d /e n ro ll e d ( a ll c o u rs e s ) GEIN http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 73 Figure 6. Performance all courses vs Performance at the first course in the case of GEIN students. Figure 7. ECTS passed/ECTS enrolled at first course vs average mark for the case of GEED students. When looking at GEED degree (Figures 7, 8 and 9), the patterns observed are very similar to those in GEEI, although the zone for high marks appears in a range which begins with a slightly upper mark, around 10 over 14. In this case it must be pointed out that there is a significant set of students which present a high student performance during the first course, but whose performance decreases throughout their studies (points x=1 and y<1 in Figure 9). This result can be attributed to the fact that some students come from the previous degree Technical Architecture and in the first course a set of different credits are transferred. If the data from the first years (2009, 2010) is eliminated, this trend becomes less significant. 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 ECTS passed/enrolled (1st course) E C T S p a s s e d /e n ro ll e d ( a ll c o u rs e s ) GEIN 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 Average mark E C T S p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) GEED http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 74 Figure 8. ECTS passed/ECTS enrolled vs average mark for the case of GEED students. Figure 9. Performance all courses vs Performance at the first course in the case of GEED students. When considering GEAM degree (Figure 10), the results reflect almost the same trends as the previously discussed, with the exception of the existence of an important set of students with low and medium marks in the admission exam who later show a high level of performance. It might be due to the vocational character of the students in this degree. 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 Average mark E C T S p a s s e d /e n ro ll e d ( a ll c o u rs e s ) GEED 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 ECTS passed/enrolled (1st course) E C T S p a s s e d /e n ro ll e d ( a ll c o u rs e s ) GEED http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 75 Figure 10. ECTS passed/ECTS enrolled in the first course vs average mark for the case of GEAM students. Finally, in the cases of GTTT (Figure 11) and GMAT (Figure 12) degrees, very similar behaviors are also observed. These degrees have a smaller number of students than all the other degrees in our university. In the case of GTTT degree a similar result as in the case of GEIN degree is observed, that is a set of students with very low performance and with medium mark of access exam. The reason is probably the same. Figure 11. ECTS passed/ECTS enrolled in the first course vs average mark for the case of GTTT students. 4 6 8 10 12 0 0.2 0.4 0.6 0.8 1 Average mark E C T S p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) GEAM 4 6 8 10 12 14 0 0.2 0.4 0.6 0.8 1 Average mark E C T S p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) GTTT http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 76 Figure 12. ECTS passed/ECTS enrolled in the first course vs average mark for the case of GMAT students. 4. Conclusions There is a high level of difficulty in degrees related to engineering, which implies higher dropout rates and lower student performance than in other fields. For this reason, it is very important to analyze the academic progress of our students in relation to their academic background, average mark, and other significant data. In this work, the different degrees taught at the Escola Politècnica Superior of the Universitat de les Illes Balears have been analyzed. The obtained results show similar trends in all the degrees, although some individual particularities are highlighted. From the results obtained, a correlation between student performance in the degree and mark of access exam is only observed in two cases. First, in the case of students presenting a low mark in the access exam (correlated with low performances) and second, in the case of students that have a high mark in the access exam (correlated with high performances). There is no clear correlation in the zone of medium marks. The statement “the poor performance of engineering students at the EPS is because they do not have good average mark” is partially right. The correlation between the average 4 6 8 10 12 0 0.2 0.4 0.6 0.8 1 Average mark E C T S p a s s e d /e n ro ll e d ( 1 s t c o u rs e ) GMAT http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 77 mark and the performance presented in this paper is significant just in a reduced zone of marks, that is, low marks and very high marks. On the other hand, there is an important correlation between first course and degree performances. The performance in the first course could be used as a predictive parameter in the student progress throughout their studies, and could allow us to propose improvement plans for the students presenting some difficulties. As a future work, we will proceed to evaluate other parameters which could have some influence in the students’ progress such as the students’ background (high school, professional formation …) in order to obtain more information about it. 5. References [1] Guía de Autoevaluación: renovación de la acreditación de títulos oficiales de Grado, Máster y Doctorado. Programa ACREDITA. Agencia Nacional de Evaluación de la Calidad y Acreditación. ANECA 18/06/2014 [2] Programa ACREDITA PLUS. Guía de evaluación para la renovación de la acreditación y la obtención del sello EUR-ACE para títulos oficiales de Grado y de Máster en Ingeniería. Agencia Nacional de Evaluación de la Calidad y Acreditación. ANECA 05/02/2015 [3] J. López et alt. “Resultados del 2º año de grado en la EUETIB en función de la titulación y la nota de selectividad de los estudiantes”. XXI Congreso Universitario de Innovación Educativa en las Enseñanzas Técnicas CUIEET2013, Valencia, 2013 [4] J. López et alt. “Resultados de la fase inicial de los grados de ingeniería industrial de la EUETIB en función de la procedencia de los estudiantes y su nota de corte”. XX Congreso Universitario de Innovación Educativa en las Enseñanzas Técnicas CUIEET2012, Las Palmas de Gran Canaria, 2012 http://dx.doi.org/10.4995/muse.2016.4693 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.4693 Social and Technological Sciences EISSN: 2341-2593 Roca et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 65-78 | 78 [5] J. López et alt. “Correlación entre la nota de ingreso a los estudios de grado de ámbito industrial y los resultados académicos obtenidos en el primer curso”. XIX Congreso Universitario de Innovación Educativa en las Enseñanzas Técnicas CUIEET2011, Barcelona, 2011 [6] J. López et alt. “Evolución de los estudiantes de segundo curso de la EUETIB referenciado ala nota de primer año de carrera y a la de selectividad, análisis de resultados por titulación”. XXII Congreso Universitario de Innovación Educativa en las Enseñanzas Técnicas CUIEET2014, Almadén (Ciudad Real), 2014 http://dx.doi.org/10.4995/muse.2016.4693